How to get BioBERT embeddings
Question:
I have field within a pandas dataframe with a text field for which I want to generate BioBERT embeddings. Is there a simple way with which I can generate the vector embeddings? I want to use them within another model.
here is a hypothetical sample of the data frame
Visit Code
Problem Assessment
1234
ge reflux working diagnosis well
4567
medication refill order working diagnosis note called in brand benicar 5mg qd 30 prn refill
I have tried this package, but receive an error upon installation
https://pypi.org/project/biobert-embedding
Error:
Collecting biobert-embedding
Using cached biobert-embedding-0.1.2.tar.gz (4.8 kB)
ERROR: Could not find a version that satisfies the requirement torch==1.2.0 (from biobert-embedding) (from versions: 0.1.2, 0.1.2.post1, 0.1.2.post2, 1.7.1)
ERROR: No matching distribution found for torch==1.2.0 (from biobert-embedding)
Any help is GREATLY appreciated!
Answers:
Try to install it as follows:
pip install biobert-embedding==0.1.2 torch==1.2.0 -f https://download.pytorch.org/whl/torch_stable.html
I extended your sample dataframe to illustrate how you can now calculate the sentence vectors for your problem assessments
and use these to calculate for example the cosine similarity between similar visit codes
.
>>> from biobert_embedding.embedding import BiobertEmbedding
>>> from scipy.spatial import distance
>>> import pandas as pd
>>> data = {'Visit Code': [1234, 1235, 4567, 4568],
'Problem Assessment': ['ge reflux working diagnosis well',
'other reflux diagnosis poor',
'medication refill order working diagnosis note called in brand benicar 5mg qd 30 prn refill',
'medication must be refilled diagnosis note called in brand Olmesartan 10mg qd 40 prn refill']}
>>> df = pd.DataFrame(data)
>>> df
Visit Code
Problem Assessment
0
1234
ge reflux working diagnosis well
1
1234
other reflux diagnosis poor
2
4567
medication refill order working diagnosis note called in brand benicar 5mg qd 30 prn refill
3
4567
medication must be refilled diagnosis note called in brand Olmesartan 10mg qd 40 prn refill
>>> biobert = BiobertEmbedding()
>>> df['sentence embedding'] = df['Problem Assessment'].apply(lambda sentence: biobert.sentence_vector(sentence))
>>> df
Visit Code
Problem Assessment
sentence embedding
0
1234
ge reflux working diagnosis well
tensor([ 2.7189e-01, -1.6195e-01, 5.8270e-02, -3.2730e-01, 7.5583e-02, …
1
1234
other reflux diagnosis poor
tensor([ 1.6971e-01, -2.1405e-01, 3.4427e-02, -2.3090e-01, 1.6007e-02, …
2
4567
medication refill order working diagnosis note called in brand benicar 5mg qd 30 prn refill
tensor([ 1.5370e-01, -3.9875e-01, 2.0089e-01, 4.1506e-02, 6.9854e-02, …
3
4567
medication must be refilled diagnosis note called in brand Olmesartan 10mg qd 40 prn refill
tensor([ 2.2128e-01, -2.0283e-01, 2.2194e-01, 9.1156e-02, 1.1620e-01, …
>>> df.groupby('Visit Code')['sentence embedding'].apply(lambda sentences: 1 - distance.cosine(sentences.values) )
Visit Code
1234 0.950492
4567 0.969715
Name: sentence embedding, dtype: float64
We can see that, as expected, the similar sentences lie very close together
I was also able to download biobert-embedding with the above answer: pip install biobert-embedding==0.1.2 torch==1.2.0 -f https://download.pytorch.org/whl/torch_stable.html
However, one minor change I had to make, I already had torch 1.12.0, so I got an error with the torch 1.2.0.
If months or years have gone by and torch has updated release, I’d first
pip install pytorch
that will install latest version, and then go with the torch version you have which mine is
pip install biobert-embedding==0.1.2 torch==1.12.0 -f https://download.pytorch.org/whl/torch_stable.html
I have field within a pandas dataframe with a text field for which I want to generate BioBERT embeddings. Is there a simple way with which I can generate the vector embeddings? I want to use them within another model.
here is a hypothetical sample of the data frame
Visit Code | Problem Assessment |
---|---|
1234 | ge reflux working diagnosis well |
4567 | medication refill order working diagnosis note called in brand benicar 5mg qd 30 prn refill |
I have tried this package, but receive an error upon installation
https://pypi.org/project/biobert-embedding
Error:
Collecting biobert-embedding
Using cached biobert-embedding-0.1.2.tar.gz (4.8 kB)
ERROR: Could not find a version that satisfies the requirement torch==1.2.0 (from biobert-embedding) (from versions: 0.1.2, 0.1.2.post1, 0.1.2.post2, 1.7.1)
ERROR: No matching distribution found for torch==1.2.0 (from biobert-embedding)
Any help is GREATLY appreciated!
Try to install it as follows:
pip install biobert-embedding==0.1.2 torch==1.2.0 -f https://download.pytorch.org/whl/torch_stable.html
I extended your sample dataframe to illustrate how you can now calculate the sentence vectors for your problem assessments
and use these to calculate for example the cosine similarity between similar visit codes
.
>>> from biobert_embedding.embedding import BiobertEmbedding
>>> from scipy.spatial import distance
>>> import pandas as pd
>>> data = {'Visit Code': [1234, 1235, 4567, 4568],
'Problem Assessment': ['ge reflux working diagnosis well',
'other reflux diagnosis poor',
'medication refill order working diagnosis note called in brand benicar 5mg qd 30 prn refill',
'medication must be refilled diagnosis note called in brand Olmesartan 10mg qd 40 prn refill']}
>>> df = pd.DataFrame(data)
>>> df
Visit Code | Problem Assessment | |
---|---|---|
0 | 1234 | ge reflux working diagnosis well |
1 | 1234 | other reflux diagnosis poor |
2 | 4567 | medication refill order working diagnosis note called in brand benicar 5mg qd 30 prn refill |
3 | 4567 | medication must be refilled diagnosis note called in brand Olmesartan 10mg qd 40 prn refill |
>>> biobert = BiobertEmbedding()
>>> df['sentence embedding'] = df['Problem Assessment'].apply(lambda sentence: biobert.sentence_vector(sentence))
>>> df
Visit Code | Problem Assessment | sentence embedding | |
---|---|---|---|
0 | 1234 | ge reflux working diagnosis well | tensor([ 2.7189e-01, -1.6195e-01, 5.8270e-02, -3.2730e-01, 7.5583e-02, … |
1 | 1234 | other reflux diagnosis poor | tensor([ 1.6971e-01, -2.1405e-01, 3.4427e-02, -2.3090e-01, 1.6007e-02, … |
2 | 4567 | medication refill order working diagnosis note called in brand benicar 5mg qd 30 prn refill | tensor([ 1.5370e-01, -3.9875e-01, 2.0089e-01, 4.1506e-02, 6.9854e-02, … |
3 | 4567 | medication must be refilled diagnosis note called in brand Olmesartan 10mg qd 40 prn refill | tensor([ 2.2128e-01, -2.0283e-01, 2.2194e-01, 9.1156e-02, 1.1620e-01, … |
>>> df.groupby('Visit Code')['sentence embedding'].apply(lambda sentences: 1 - distance.cosine(sentences.values) )
Visit Code
1234 0.950492
4567 0.969715
Name: sentence embedding, dtype: float64
We can see that, as expected, the similar sentences lie very close together
I was also able to download biobert-embedding with the above answer: pip install biobert-embedding==0.1.2 torch==1.2.0 -f https://download.pytorch.org/whl/torch_stable.html
However, one minor change I had to make, I already had torch 1.12.0, so I got an error with the torch 1.2.0.
If months or years have gone by and torch has updated release, I’d first
pip install pytorch
that will install latest version, and then go with the torch version you have which mine is
pip install biobert-embedding==0.1.2 torch==1.12.0 -f https://download.pytorch.org/whl/torch_stable.html