How to write code to autocomplete words and sentences?
Question:
I’d like to write code that does autocompletion in the Linux terminal. The code should work as follows.
It has a list of strings (e.g. "hello, "hi", "how are you", "goodbye", "great", …).
In the terminal, the user will start typing and when there is some match possibility, he gets the hint for possible strings, from which he can choose (similarly as in vim editor or google incremental search).
e.g. he starts typing "h", and he gets the hint
h"ello"
_ "i"
_"ow are you"
And better yet would be if it would complete words not only from the beginning, but from an arbitrary part of the string.
Answers:
I guess you will need to get a key pressed by the user.
You can achieve it (without pressing enter) with a method like this:
import termios, os, sys
def getkey():
fd = sys.stdin.fileno()
old = termios.tcgetattr(fd)
new = termios.tcgetattr(fd)
new[3] = new[3] & ~termios.ICANON & ~termios.ECHO
new[6][termios.VMIN] = 1
new[6][termios.VTIME] = 0
termios.tcsetattr(fd, termios.TCSANOW, new)
c = None
try:
c = os.read(fd, 1)
finally:
termios.tcsetattr(fd, termios.TCSAFLUSH, old)
return c
Then, if this key is a tab key (for example, that’s something you need to implement), then display all possibilities to user. If that’s any other key, print it on stdout.
Oh, of course you will need to have getkey() looped in a while, as long as the user hits enter. You can also get a method like raw_input, that will get the whole word sign by sign, or display all the possibilities, when you hit a tab.
At least that’s the item, you can start with. If you achieve any other problems, than write about them.
EDIT 1:
The get_word method can look like this:
def get_word():
s = ""
while True:
a = getkey()
if a == "n":
break
elif a == "t":
print "all possibilities"
else:
s += a
return s
word = get_word()
print word
The issue I’m occuring right now is the way to display a sign, you have just entered without any enteres and spaces, what both print a and print a, does.
(I’m aware this isn’t exactly what you’re asking for, but) If you’re happy with the auto-completion/suggestions appearing on TAB (as used in many shells), then you can quickly get up and running using the readline module.
Here’s a quick example based on Doug Hellmann’s PyMOTW writeup on readline.
import readline
class MyCompleter(object): # Custom completer
def __init__(self, options):
self.options = sorted(options)
def complete(self, text, state):
if state == 0: # on first trigger, build possible matches
if text: # cache matches (entries that start with entered text)
self.matches = [s for s in self.options
if s and s.startswith(text)]
else: # no text entered, all matches possible
self.matches = self.options[:]
# return match indexed by state
try:
return self.matches[state]
except IndexError:
return None
completer = MyCompleter(["hello", "hi", "how are you", "goodbye", "great"])
readline.set_completer(completer.complete)
readline.parse_and_bind('tab: complete')
input = raw_input("Input: ")
print "You entered", input
This results in the following behaviour (<TAB> representing a the tab key being pressed):
Input: <TAB><TAB>
goodbye great hello hi how are you
Input: h<TAB><TAB>
hello hi how are you
Input: ho<TAB>ow are you
In the last line (HOTAB entered), there is only one possible match and the whole sentence “how are you” is auto completed.
Check out the linked articles for more information on readline.
“And better yet would be if it would complete words not only from the beginning … completion from arbitrary part of the string.”
This can be achieved by simply modifying the match criteria in the completer function, ie. from:
self.matches = [s for s in self.options
if s and s.startswith(text)]
to something like:
self.matches = [s for s in self.options
if text in s]
This will give you the following behaviour:
Input: <TAB><TAB>
goodbye great hello hi how are you
Input: o<TAB><TAB>
goodbye hello how are you
Updates: using the history buffer (as mentioned in comments)
A simple way to create a pseudo-menu for scrolling/searching is to load the keywords into the history buffer. You will then be able to scroll through the entries using the up/down arrow keys as well as use Ctrl+R to perform a reverse-search.
To try this out, make the following changes:
keywords = ["hello", "hi", "how are you", "goodbye", "great"]
completer = MyCompleter(keywords)
readline.set_completer(completer.complete)
readline.parse_and_bind('tab: complete')
for kw in keywords:
readline.add_history(kw)
input = raw_input("Input: ")
print "You entered", input
When you run the script, try typing Ctrl+r followed by a. That will return the first match that contains “a”. Enter Ctrl+r again for the next match. To select an entry, press ENTER.
Also try using the UP/DOWN keys to scroll through the keywords.
For those (like me) that end up here searching for autocomplete in the interpreter:
This involves creating a file .pythonrc, modifying .bashrc and an import sys you have to import every time you launch the Python interpreter.
I wonder if the latter can be automated for even more win.
To enable autocomplete in a Python shell, type this:
import rlcompleter, readline
readline.parse_and_bind('tab:complete')
(thanks to http://blog.e-shell.org/221)
Steps:
-
Create a file .pythonrc in home directory by this command:
vi .pythonrc
-
Enter this content:
import rlcompleter, readline
readline.parse_and_bind('tab:complete')
-
Close the file
-
Now run
echo "export PYTHONSTARTUP=~/.pythonrc" >> ~/.bashrc
-
Restart the terminal
You may want to checkout fast-autocomplete: https://github.com/seperman/fast-autocomplete
It has a demo mode that you can type and get results as you type: https://zepworks.com/posts/you-autocomplete-me/#part-6-demo
It is very easy to use:
>>> from fast_autocomplete import AutoComplete
>>> words = {'book': {}, 'burrito': {}, 'pizza': {}, 'pasta':{}}
>>> autocomplete = AutoComplete(words=words)
>>> autocomplete.search(word='b', max_cost=3, size=3)
[['book'], ['burrito']]
>>> autocomplete.search(word='bu', max_cost=3, size=3)
[['burrito']]
>>> autocomplete.search(word='barrito', max_cost=3, size=3) # mis-spelling
[['burrito']]
Disclaimer: I wrote fast-autocomplete.
I’d like to write code that does autocompletion in the Linux terminal. The code should work as follows.
It has a list of strings (e.g. "hello, "hi", "how are you", "goodbye", "great", …).
In the terminal, the user will start typing and when there is some match possibility, he gets the hint for possible strings, from which he can choose (similarly as in vim editor or google incremental search).
e.g. he starts typing "h", and he gets the hint
h"ello"
_ "i"
_"ow are you"
And better yet would be if it would complete words not only from the beginning, but from an arbitrary part of the string.
I guess you will need to get a key pressed by the user.
You can achieve it (without pressing enter) with a method like this:
import termios, os, sys
def getkey():
fd = sys.stdin.fileno()
old = termios.tcgetattr(fd)
new = termios.tcgetattr(fd)
new[3] = new[3] & ~termios.ICANON & ~termios.ECHO
new[6][termios.VMIN] = 1
new[6][termios.VTIME] = 0
termios.tcsetattr(fd, termios.TCSANOW, new)
c = None
try:
c = os.read(fd, 1)
finally:
termios.tcsetattr(fd, termios.TCSAFLUSH, old)
return c
Then, if this key is a tab key (for example, that’s something you need to implement), then display all possibilities to user. If that’s any other key, print it on stdout.
Oh, of course you will need to have getkey() looped in a while, as long as the user hits enter. You can also get a method like raw_input, that will get the whole word sign by sign, or display all the possibilities, when you hit a tab.
At least that’s the item, you can start with. If you achieve any other problems, than write about them.
EDIT 1:
The get_word method can look like this:
def get_word():
s = ""
while True:
a = getkey()
if a == "n":
break
elif a == "t":
print "all possibilities"
else:
s += a
return s
word = get_word()
print word
The issue I’m occuring right now is the way to display a sign, you have just entered without any enteres and spaces, what both print a and print a, does.
(I’m aware this isn’t exactly what you’re asking for, but) If you’re happy with the auto-completion/suggestions appearing on TAB (as used in many shells), then you can quickly get up and running using the readline module.
Here’s a quick example based on Doug Hellmann’s PyMOTW writeup on readline.
import readline
class MyCompleter(object): # Custom completer
def __init__(self, options):
self.options = sorted(options)
def complete(self, text, state):
if state == 0: # on first trigger, build possible matches
if text: # cache matches (entries that start with entered text)
self.matches = [s for s in self.options
if s and s.startswith(text)]
else: # no text entered, all matches possible
self.matches = self.options[:]
# return match indexed by state
try:
return self.matches[state]
except IndexError:
return None
completer = MyCompleter(["hello", "hi", "how are you", "goodbye", "great"])
readline.set_completer(completer.complete)
readline.parse_and_bind('tab: complete')
input = raw_input("Input: ")
print "You entered", input
This results in the following behaviour (<TAB> representing a the tab key being pressed):
Input: <TAB><TAB>
goodbye great hello hi how are you
Input: h<TAB><TAB>
hello hi how are you
Input: ho<TAB>ow are you
In the last line (HOTAB entered), there is only one possible match and the whole sentence “how are you” is auto completed.
Check out the linked articles for more information on readline.
“And better yet would be if it would complete words not only from the beginning … completion from arbitrary part of the string.”
This can be achieved by simply modifying the match criteria in the completer function, ie. from:
self.matches = [s for s in self.options
if s and s.startswith(text)]
to something like:
self.matches = [s for s in self.options
if text in s]
This will give you the following behaviour:
Input: <TAB><TAB>
goodbye great hello hi how are you
Input: o<TAB><TAB>
goodbye hello how are you
Updates: using the history buffer (as mentioned in comments)
A simple way to create a pseudo-menu for scrolling/searching is to load the keywords into the history buffer. You will then be able to scroll through the entries using the up/down arrow keys as well as use Ctrl+R to perform a reverse-search.
To try this out, make the following changes:
keywords = ["hello", "hi", "how are you", "goodbye", "great"]
completer = MyCompleter(keywords)
readline.set_completer(completer.complete)
readline.parse_and_bind('tab: complete')
for kw in keywords:
readline.add_history(kw)
input = raw_input("Input: ")
print "You entered", input
When you run the script, try typing Ctrl+r followed by a. That will return the first match that contains “a”. Enter Ctrl+r again for the next match. To select an entry, press ENTER.
Also try using the UP/DOWN keys to scroll through the keywords.
For those (like me) that end up here searching for autocomplete in the interpreter:
This involves creating a file .pythonrc, modifying .bashrc and an import sys you have to import every time you launch the Python interpreter.
I wonder if the latter can be automated for even more win.
To enable autocomplete in a Python shell, type this:
import rlcompleter, readline
readline.parse_and_bind('tab:complete')
(thanks to http://blog.e-shell.org/221)
Steps:
-
Create a file .pythonrc in home directory by this command:
vi .pythonrc -
Enter this content:
import rlcompleter, readline readline.parse_and_bind('tab:complete') -
Close the file
-
Now run
echo "export PYTHONSTARTUP=~/.pythonrc" >> ~/.bashrc -
Restart the terminal
You may want to checkout fast-autocomplete: https://github.com/seperman/fast-autocomplete
It has a demo mode that you can type and get results as you type: https://zepworks.com/posts/you-autocomplete-me/#part-6-demo
It is very easy to use:
>>> from fast_autocomplete import AutoComplete
>>> words = {'book': {}, 'burrito': {}, 'pizza': {}, 'pasta':{}}
>>> autocomplete = AutoComplete(words=words)
>>> autocomplete.search(word='b', max_cost=3, size=3)
[['book'], ['burrito']]
>>> autocomplete.search(word='bu', max_cost=3, size=3)
[['burrito']]
>>> autocomplete.search(word='barrito', max_cost=3, size=3) # mis-spelling
[['burrito']]
Disclaimer: I wrote fast-autocomplete.