PyCharm intellisense for boto3
Question:
having problems seeing full intellisense (code completion) options in PyCharm.
working with python 3.4 on Windows.
the suggests are partially working:
import boto3
s = boto3.Session() (boto3. will bring up list of methods/params of object boto3)
ec2 = s.resource('ec2') (resource is a suggested method!)
ec2. <<<< this brings up nothing. For some reason PyCharm cant detect that ec2 object would have
while I can work off documentation alone, intellisense is just such a nice feature to have!
ive had similar problems getting it to complete lxml syntax but I thought that was because I had to install lxml directly as a binary (too many hoops to jump through on windows to install it via pip)
Anyone else encounter similar problems?
While we are here,
I see a lot of different libraries around using awscli with python: boto, boto3, troposphere etc. What are some advantages of using one over the other. Amazon states that boto3 is the prefered method over boto but for my usage of starting/stopping ec2 instances could be easily done with older boto.
Answers:
This is happening because all of the methods on the boto3 clients and resource objects are generated at runtime based on a JSON file that describes what operations the service supports. Pycharm would have to have specific knowledge about this process in order to auto complete method names.
For your second question, boto3 is the official AWS SDK for Python. One of the main advantages of boto3 is that because of this JSON model driven process that describes the AWS APIs, most new service features only require a simple model update. This means API updates happen in a quick, consistent, and reliable manner.
But if you’re using boto in existing code and it’s working for you, feel free to keep using it. You can always install boto3 along side boto if you need to pull in new functionality.
I was frustrated with the same issue. So I decided to parse boto3 documentation and generate wrapper classes from the documentation. Here is the link to project
https://github.com/gehadshaat/pyboto3
To install it
pip install pyboto3
To use it
import boto3
s3 = boto3.client('s3')
""" :type : pyboto3.s3 """
# s3. -> will give you autocomplete for s3 methods in pycharm
Make sure that you first:
- Install pyboto3 -> pip install pyboto3 | pip3.x install pyboto3
- Check your interpreter settings and verify that you see pyboto3 on the list
- Do a File -> Invalidate Caches/Restart
After Pycharm restarts you should see intellisense working in your favor and all of the available methods for the service (in the case above s3) you are trying to use available to you!
Probably not an official method, but I did find something that works.
In PyCharm, open python console (tools/python console). The console will also have variable list on the right side. If you initialize a resource object on the console, it will have its sub objects listed in variable object tree. Some limited intellisense as well.
The way I started doing it, is writing code right into the interpreter using variable watch window as a cheat sheet. Once code is written, I copy/paste it into the actual script file. Clunky…
I love boto3, but I was also frustrated that every time I want to make a simple ad-hoc request I have to open boto3 documentation. So I wrote autoboto:
https://pypi.org/project/autoboto/
It doesn’t just auto-complete. It also returns dataclasses which means that you don’t have to look up the names of attributes of the returned objects. PyCharm will tell you what is available.
At the moment, it’s also probably very slow because of all the generic serialisation and deserialisation.
The room’s getting a little crowded here, but I also have created a boto3 typing solution (GitHub link), boto3_type_annotations. I took the pyboto3 approach and parsed the docstrings of service objects and then programmatically wrote class definitions for them and annotated arguments and return types with the typing module. Unlike pyboto3 I created everything including service resources, paginators, and waiters. There’s also an option where I left the docstrings in, so PyCharm’s quick documentation will work. But fair warning, that package is really big.
# Without docs
pip install boto3_type_annotations
# Or with docs
pip install boto3_type_annotations_with_docs
Make sure that you:
- Install pyboto3 -> pip install pyboto3 | pip3.x install pyboto3
- Check your interpreter settings and verify that you see pyboto3 on the list
- Do a File -> Invalidate Caches/Restart
After Pycharm restarts you should see intellisense working in your favor and all of the available methods for the service you are trying to use available to you!
While autocomplete solutions for boto3 are being discussed I’m surprised that nobody mentioned botostubs yet. Works on any IDE and is automatically kept up-to-date.
Type annotations for boto3 1.16.62 compatible with VSCode, PyCharm, Emacs, Sublime Text, mypy, pyright and other tools.
This works for me. If you are using python3
python3 -m pip install boto3-stubs
python3 -m pip install 'boto3-stubs[essential]'
UPDATES 2023-03-24
aws-samples just release an open source boto_session_manager, it has a thin integration layer with boto_stubs, and it has explicit type hint out of the box. With this, it can pop up auto complete and argument hint. Also, you can one click to jump to the detailed document.
having problems seeing full intellisense (code completion) options in PyCharm.
working with python 3.4 on Windows.
the suggests are partially working:
import boto3
s = boto3.Session() (boto3. will bring up list of methods/params of object boto3)
ec2 = s.resource('ec2') (resource is a suggested method!)
ec2. <<<< this brings up nothing. For some reason PyCharm cant detect that ec2 object would have
while I can work off documentation alone, intellisense is just such a nice feature to have!
ive had similar problems getting it to complete lxml syntax but I thought that was because I had to install lxml directly as a binary (too many hoops to jump through on windows to install it via pip)
Anyone else encounter similar problems?
While we are here,
I see a lot of different libraries around using awscli with python: boto, boto3, troposphere etc. What are some advantages of using one over the other. Amazon states that boto3 is the prefered method over boto but for my usage of starting/stopping ec2 instances could be easily done with older boto.
This is happening because all of the methods on the boto3 clients and resource objects are generated at runtime based on a JSON file that describes what operations the service supports. Pycharm would have to have specific knowledge about this process in order to auto complete method names.
For your second question, boto3 is the official AWS SDK for Python. One of the main advantages of boto3 is that because of this JSON model driven process that describes the AWS APIs, most new service features only require a simple model update. This means API updates happen in a quick, consistent, and reliable manner.
But if you’re using boto in existing code and it’s working for you, feel free to keep using it. You can always install boto3 along side boto if you need to pull in new functionality.
I was frustrated with the same issue. So I decided to parse boto3 documentation and generate wrapper classes from the documentation. Here is the link to project
https://github.com/gehadshaat/pyboto3
To install it
pip install pyboto3
To use it
import boto3
s3 = boto3.client('s3')
""" :type : pyboto3.s3 """
# s3. -> will give you autocomplete for s3 methods in pycharm
Make sure that you first:
- Install pyboto3 -> pip install pyboto3 | pip3.x install pyboto3
- Check your interpreter settings and verify that you see pyboto3 on the list
- Do a File -> Invalidate Caches/Restart
After Pycharm restarts you should see intellisense working in your favor and all of the available methods for the service (in the case above s3) you are trying to use available to you!
Probably not an official method, but I did find something that works.
In PyCharm, open python console (tools/python console). The console will also have variable list on the right side. If you initialize a resource object on the console, it will have its sub objects listed in variable object tree. Some limited intellisense as well.
The way I started doing it, is writing code right into the interpreter using variable watch window as a cheat sheet. Once code is written, I copy/paste it into the actual script file. Clunky…
I love boto3, but I was also frustrated that every time I want to make a simple ad-hoc request I have to open boto3 documentation. So I wrote autoboto:
https://pypi.org/project/autoboto/
It doesn’t just auto-complete. It also returns dataclasses which means that you don’t have to look up the names of attributes of the returned objects. PyCharm will tell you what is available.
At the moment, it’s also probably very slow because of all the generic serialisation and deserialisation.
The room’s getting a little crowded here, but I also have created a boto3 typing solution (GitHub link), boto3_type_annotations. I took the pyboto3 approach and parsed the docstrings of service objects and then programmatically wrote class definitions for them and annotated arguments and return types with the typing module. Unlike pyboto3 I created everything including service resources, paginators, and waiters. There’s also an option where I left the docstrings in, so PyCharm’s quick documentation will work. But fair warning, that package is really big.
# Without docs
pip install boto3_type_annotations
# Or with docs
pip install boto3_type_annotations_with_docs
Make sure that you:
- Install pyboto3 -> pip install pyboto3 | pip3.x install pyboto3
- Check your interpreter settings and verify that you see pyboto3 on the list
- Do a File -> Invalidate Caches/Restart
After Pycharm restarts you should see intellisense working in your favor and all of the available methods for the service you are trying to use available to you!
While autocomplete solutions for boto3 are being discussed I’m surprised that nobody mentioned botostubs yet. Works on any IDE and is automatically kept up-to-date.
Type annotations for boto3 1.16.62 compatible with VSCode, PyCharm, Emacs, Sublime Text, mypy, pyright and other tools.
This works for me. If you are using python3
python3 -m pip install boto3-stubs
python3 -m pip install 'boto3-stubs[essential]'
UPDATES 2023-03-24
aws-samples just release an open source boto_session_manager, it has a thin integration layer with boto_stubs, and it has explicit type hint out of the box. With this, it can pop up auto complete and argument hint. Also, you can one click to jump to the detailed document.