tools to aid in browsing/following (large) python projects' source code
Question:
A specific example: becoming familiar with django’s project source code (core, contrib, utils, etc.). Example of a useful tool: ctags – it allows you to “jump” to the file+location where a function/method is defined. Wondering about other tools that developers use (example: is there a tool that given a function x(), lists the functions that call x() and that are called by x()?). Thanks.
Edit: added an answer with an aggregate of tools mentioned so far in other answers
Answers:
You can maybe try cscope! Wikipedia says that
cscope is often used to search content within C or C++ files, but it can be used to search for content in other languages such as Java, Python, PHP and Perl.[citation needed]
And you can also dig in this project.
I think Komodo Edit and PyDev allows you to jump to python function defs.
This is subjective so I think it should probably be a community wiki. That said, the best thing you can probably do to make browsing large projects is to be familiar with hotkeys provided in your favourite IDE. Using the keyboard to browse through large source code is much easier than manually scrolling through text, highlighting text and fumbling through an IDE with a mouse.
Document it as you go. Leave trails, improve the structure, and keep notes. By the time you’ve found you way around the enter codebase, you’ll have a good map.
Many (or even most, I should say) IDE’s help you in this by enabling you do go to variable and function definitions, often by just Ctrl+click, or showing you class overviews where you can see all methods and attributes a class has including those inherited, and letting you go to their definition, etc, etc, etc. I can’t recommend such a tool highly enough, it’s very time-saving for development.
I personally use WingIDE, which is excellent and has all these features, but you should also check out KomodoEdit and Eclipse+PyDev. There maybe more that I don’t know of, and it’s fully possible that vim and emacs have some sort of plugins for this.
The following is an aggregate of tools mentioned in other answers…
cscope
http://cscope.sourceforge.net/
wikipedia entry: http://en.wikipedia.org/wiki/Cscope
cscope is a console mode or text-based graphical interface … It is often used on very large projects to find source code, functions, declarations, definitions and regular expressions given a text string.
pycscope
http://pypi.python.org/pypi/pycscope/
generates a cscope index of Python source trees
ctags and exuberant ctags
http://ctags.sourceforge.net/ctags.html
wikipedia entry: http://en.wikipedia.org/wiki/Ctags
Ctags is a program that generates an index (or tag) file of names found in source and header files of various programming languages. Depending on the language, functions, variables, class members, macros and so on may be indexed. These tags allow definitions to be quickly and easily located by a text editor or other utility.
Eclipse:
wikipedia entry: http://en.wikipedia.org/wiki/Eclipse_%28software%29
Eclipse is a multi-language software development platform comprising an IDE and a plug-in system to extend it. It is written primarily in Java and can be used to develop applications in Java and, by means of the various plug-ins, in other languages as well, including C, C++, COBOL, Python, Perl, PHP, and others.
PyDev
"Pydev is a plugin that enables users to use Eclipse for Python and Jython development — making Eclipse a first class Python IDE"
Komodo Edit
http://www.activestate.com/komodo_edit/
wikipedia entry: http://en.wikipedia.org/wiki/ActiveState_Komodo
Komodo Edit is a free text editor for dynamic programming languages introduced in January 2007. With the release of version 4.3, Komodo Edit is built on top of the Open Komodo project.
It was developed for programmers who need a multi-language editor with broad functionality, but not the features of an IDE, like debugging, DOM viewer, interactive shells, and source code control integration.
Prashanth’s call graph (visualization) tool
Just thought I’d share a link to an interesting small fun script I’ve found long time ago, that draws a graph of function calls. It works only for simple cases, so "as is" it’s more fun than useful.
rope/ropemacs
http://rope.sourceforge.net/ropemacs.html
Ropemacs is a plugin for performing python refactorings in emacs. It uses rope library and pymacs.
http://www.enigmacurry.com/2008/05/09/emacs-as-a-powerful-python-ide/
Wing IDE
Wing IDE has goto-definition, find uses, a source browser, refactoring, and other code intelligence features that should help. Another good way to understand unfamiliar Python code is to set a breakpoint, run to it in the debugger, and then go up and down the stack. In Wing Professional you can also use the Debug Probe to interact with and try out things in the debug runtime state (it’s a Python shell that runs in the context of the current debug stack frame).
is there a tool that given a function x(), lists the functions that call x() and that are called by x()?
Just thought I’d share a link to an interesting small fun script I’ve found long time ago, that draws a graph of function calls. It works only for simple cases, so "as is" it’s more fun than useful.
For normal Python development personally I use GNU Emacs with rope/ropemacs (found a video showing the features) and sometimes Eclipse with PyDev.
You should notice that cscope targets only the UNIX, Linux OSs.
A specific example: becoming familiar with django’s project source code (core, contrib, utils, etc.). Example of a useful tool: ctags – it allows you to “jump” to the file+location where a function/method is defined. Wondering about other tools that developers use (example: is there a tool that given a function x(), lists the functions that call x() and that are called by x()?). Thanks.
Edit: added an answer with an aggregate of tools mentioned so far in other answers
You can maybe try cscope! Wikipedia says that
cscope is often used to search content within C or C++ files, but it can be used to search for content in other languages such as Java, Python, PHP and Perl.[citation needed]
And you can also dig in this project.
I think Komodo Edit and PyDev allows you to jump to python function defs.
This is subjective so I think it should probably be a community wiki. That said, the best thing you can probably do to make browsing large projects is to be familiar with hotkeys provided in your favourite IDE. Using the keyboard to browse through large source code is much easier than manually scrolling through text, highlighting text and fumbling through an IDE with a mouse.
Document it as you go. Leave trails, improve the structure, and keep notes. By the time you’ve found you way around the enter codebase, you’ll have a good map.
Many (or even most, I should say) IDE’s help you in this by enabling you do go to variable and function definitions, often by just Ctrl+click, or showing you class overviews where you can see all methods and attributes a class has including those inherited, and letting you go to their definition, etc, etc, etc. I can’t recommend such a tool highly enough, it’s very time-saving for development.
I personally use WingIDE, which is excellent and has all these features, but you should also check out KomodoEdit and Eclipse+PyDev. There maybe more that I don’t know of, and it’s fully possible that vim and emacs have some sort of plugins for this.
The following is an aggregate of tools mentioned in other answers…
cscope
http://cscope.sourceforge.net/
wikipedia entry: http://en.wikipedia.org/wiki/Cscope
cscope is a console mode or text-based graphical interface … It is often used on very large projects to find source code, functions, declarations, definitions and regular expressions given a text string.
pycscope
http://pypi.python.org/pypi/pycscope/
generates a cscope index of Python source trees
ctags and exuberant ctags
http://ctags.sourceforge.net/ctags.html
wikipedia entry: http://en.wikipedia.org/wiki/Ctags
Ctags is a program that generates an index (or tag) file of names found in source and header files of various programming languages. Depending on the language, functions, variables, class members, macros and so on may be indexed. These tags allow definitions to be quickly and easily located by a text editor or other utility.
Eclipse:
wikipedia entry: http://en.wikipedia.org/wiki/Eclipse_%28software%29
Eclipse is a multi-language software development platform comprising an IDE and a plug-in system to extend it. It is written primarily in Java and can be used to develop applications in Java and, by means of the various plug-ins, in other languages as well, including C, C++, COBOL, Python, Perl, PHP, and others.
PyDev
"Pydev is a plugin that enables users to use Eclipse for Python and Jython development — making Eclipse a first class Python IDE"
Komodo Edit
http://www.activestate.com/komodo_edit/
wikipedia entry: http://en.wikipedia.org/wiki/ActiveState_Komodo
Komodo Edit is a free text editor for dynamic programming languages introduced in January 2007. With the release of version 4.3, Komodo Edit is built on top of the Open Komodo project.
It was developed for programmers who need a multi-language editor with broad functionality, but not the features of an IDE, like debugging, DOM viewer, interactive shells, and source code control integration.
Prashanth’s call graph (visualization) tool
Just thought I’d share a link to an interesting small fun script I’ve found long time ago, that draws a graph of function calls. It works only for simple cases, so "as is" it’s more fun than useful.
rope/ropemacs
http://rope.sourceforge.net/ropemacs.html
Ropemacs is a plugin for performing python refactorings in emacs. It uses rope library and pymacs.
http://www.enigmacurry.com/2008/05/09/emacs-as-a-powerful-python-ide/
Wing IDE
Wing IDE has goto-definition, find uses, a source browser, refactoring, and other code intelligence features that should help. Another good way to understand unfamiliar Python code is to set a breakpoint, run to it in the debugger, and then go up and down the stack. In Wing Professional you can also use the Debug Probe to interact with and try out things in the debug runtime state (it’s a Python shell that runs in the context of the current debug stack frame).
is there a tool that given a function x(), lists the functions that call x() and that are called by x()?
Just thought I’d share a link to an interesting small fun script I’ve found long time ago, that draws a graph of function calls. It works only for simple cases, so "as is" it’s more fun than useful.
For normal Python development personally I use GNU Emacs with rope/ropemacs (found a video showing the features) and sometimes Eclipse with PyDev.
You should notice that cscope targets only the UNIX, Linux OSs.