Lua as a general-purpose scripting language?
Question:
When I see Lua, the only thing I ever read is “great for embedding”, “fast”, “lightweight” and more often than anything else: “World of Warcraft” or in short “WoW”.
Why is it limited to embedding the whole thing into another application? Why not write general-purpose scripts like you do with Python or Perl?
Lua seems to be doing great in aspects like speed and memory-usage (The fastest scripting language afaik) so why is it that I never see Lua being used as a “Desktop scripting-language” to automate tasks? For example:
- Renaming a bunch of files
- Download some files from the web
- Webscraping
Is it the lack of the standard library?
Answers:
Just because it is “marketed” (in some general sense) as a special-purpose language for embedded script engines, does not mean that it is limited to that. In fact, WoW could probably just as well have chosen Python as their embedded scripting language.
It’s probably because Lua was designed as a scripting and extension language. On the official site it’s described as a powerful, fast, light-weight, embeddable scripting language. There’s nothing stopping you from writing general purpose programs for it (if I recall correctly it ships with an interpreter and compiler), but the language designers intended it to be used mainly as an embedded language (hence being light-weight and all)
Definitely a lack of standard libraries. It’s also lesser known than Python, Perl or Ruby.
Lua is a cool language, light-weight and extremely fast!
But the point is: Is performance so important for those
tasks you mentioned?
- Renaming a bunch of files
- Download some files from the web
- Webscraping
You write those programs once, and run them once, too maybe.
Why do you care about performance so much for a run-once program?
For example:
- Cost 3 hours to write a C/C++ program, to handle data once, the program will take 1 hour to run.
- Cost 30 Minute to write a Python program to handle data once, the program will take 10 hours to run.
If you choose the first, you save the time to run the program,
but you cost your time to develop the program.
On the other hand, if you choose the second, you waste time to run
the program, but you can do other things when the program is
running. How about play World of Warcraft, kill monsters
with your warlock? Eat my D.O.T! 😛
That’s it! Although Lua is not so difficult to write, everything about Lua is designed to be efficient.And what’s more, there are little modules for Lua, but there are so many modules for Python. You don’t want to port a C library for Lua just for a run-once program, do you? Instead, choose Python and use those module to achieve your task easily might be a better idea.
FYI: Actually, I have tried to use Lua to do webscraping,
but finally, I realized I do not have to care so much about language performance. The bottleneck of webscraping is
not on the performance of the language. The bottleneck is on
network I/O, HTML parsing and multitasking. All I have to do
is make sure the program works and find the bottleneck.
Finally, I chose Python rather than Lua. There is so
many excellent Python modules; I have no reason to build my
own.
According to my experience about webscraping, I chose
Twisted for network I/O and lxml for html parsing as the backend
of my webscraping program. I have wrote an article for an introduction to this technology.
The best choice to grab data from websites: Python + Twisted + lxml
Hope this is helpful.
I think the answer about it being a “marketing” thing is probably correct, along with the lack of a large set of libraries to choose from. I would like to point out another case of this: Ruby. Ruby is meant to be a general purpose scripting language. The problem is that since Ruby on Rails has risen to be so popular, it is becoming hard to find something that is unrelated to Rails. I’m afraid Lua will suffer this as well, being popular because of a few major things using it, but never able to break free of that stigma.
Lua has fewer libraries than Python. But be sure to have a look at LuaForge. It has a lot of interesting libs, like LuaCURL, wxLua or getopt.
Then, visit LuaRocks, the package management system for Lua. With it, you can search and install most mature Lua modules with dependencies. It feels like RubyGems or aptitude.
The site lua-users.org has a lot of interesting resources too, like tutorials or the Lua Wiki.
What I like about Lua is not its speed, it’s its minimal core language, flexibility and extensibility.
That said, I would probably use Python for the tasks you mentionned because of the larger community doing such things in Python.
This is a sociological question, not a programming question.
I use Lua for general-purpose scripting almost exclusively. But I had to write a few hundred lines of code so that Lua would play better with the shell. This included such tricks as
- Quoting a string so it is seen as one word by the shell
- Writing a function to capture the output of a command as in shell $(command)
- Writing a function to crawl the filesystem using the Lua posix library and expand shell globbing patterns
(For those who may be interested, I’ve left the code in my Lua drop box, which also contains some other stuff. The interesting stuff is probably in osutil in os.quote, os.runf, os.capture, and maybe os.execve. The globbing is in posixutil.lua. They both use Luiz Henrique de Figuereido’s Lua Posix library.)
To me, the extra effort is worth it because I can deal with simple syntax and great data structures. To others, a more direct connection with the shell might be preferred.
Lack of standard library. Period. Even listing all the files in a directory require a non-standard module.
There are good reasons for that (keeping strict ANSI portability, not requiring POSIX) but the result is that, for general programming, I prefer Python.
There has been a recent push to create a batteries included installation for Lua on Windows. The result can be found at the Lua for Windows project at LuaForge. It includes the interpreter and a large collection of extra modules allowing useful scripts and applications to be written and used out of the box.
I know that various Linux distros are including Lua and some modules now, and more to come.
There are also a couple of proposed module libraries under discussion in the mailing list, but the community hasn’t yet settled on one as the “official” mechanism.
I use Lua both as a scripting language and as the “main” loop of my typical application, supported by one or more DLLs containing code that was better implemented in C, or wrapping existing libraries or API functions that are needed by a particular project. Used with a GUI toolkit such as IUP or wxLua (a Lua binding for wxWindows), Lua makes writing small to mid-sized GUI applications quite pleasant.
In order for Lua to be easy to embed it has to have few dependencies and be small. That makes it poorly suited as a general purpose scripting language. Because using it as a general purpose script language would require a lot of standard libraries. But if Lua had a lot of standard libraries it would be harder to embed (due to dependencies and memory footprint.)
Lua is used in LuaTeX, a TeX extension, as an embedded language, and has gained popularity rapidly among TeX developers because of that. It is used as a scripting language for some utilities in the TeX Live distribution, be it only because now there is a luatex binary, available on all platforms, that can also be used as a Lua interpreter (with some vital modules added – slnunicode, luafilesystem, etc.) That’s very important for Windows installations, that relied on additional Unix scripting tools earlier (ActivePerl, etc.) The ConTeXt macro language uses Lua scripts extensively nowadays.
That’s admittedly a very special field 🙂 But completely unrelated to games!
When I see Lua, the only thing I ever read is “great for embedding”, “fast”, “lightweight” and more often than anything else: “World of Warcraft” or in short “WoW”.
Why is it limited to embedding the whole thing into another application? Why not write general-purpose scripts like you do with Python or Perl?
Lua seems to be doing great in aspects like speed and memory-usage (The fastest scripting language afaik) so why is it that I never see Lua being used as a “Desktop scripting-language” to automate tasks? For example:
- Renaming a bunch of files
- Download some files from the web
- Webscraping
Is it the lack of the standard library?
Just because it is “marketed” (in some general sense) as a special-purpose language for embedded script engines, does not mean that it is limited to that. In fact, WoW could probably just as well have chosen Python as their embedded scripting language.
It’s probably because Lua was designed as a scripting and extension language. On the official site it’s described as a powerful, fast, light-weight, embeddable scripting language. There’s nothing stopping you from writing general purpose programs for it (if I recall correctly it ships with an interpreter and compiler), but the language designers intended it to be used mainly as an embedded language (hence being light-weight and all)
Definitely a lack of standard libraries. It’s also lesser known than Python, Perl or Ruby.
Lua is a cool language, light-weight and extremely fast!
But the point is: Is performance so important for those
tasks you mentioned?
- Renaming a bunch of files
- Download some files from the web
- Webscraping
You write those programs once, and run them once, too maybe.
Why do you care about performance so much for a run-once program?
For example:
- Cost 3 hours to write a C/C++ program, to handle data once, the program will take 1 hour to run.
- Cost 30 Minute to write a Python program to handle data once, the program will take 10 hours to run.
If you choose the first, you save the time to run the program,
but you cost your time to develop the program.
On the other hand, if you choose the second, you waste time to run
the program, but you can do other things when the program is
running. How about play World of Warcraft, kill monsters
with your warlock? Eat my D.O.T! 😛
That’s it! Although Lua is not so difficult to write, everything about Lua is designed to be efficient.And what’s more, there are little modules for Lua, but there are so many modules for Python. You don’t want to port a C library for Lua just for a run-once program, do you? Instead, choose Python and use those module to achieve your task easily might be a better idea.
FYI: Actually, I have tried to use Lua to do webscraping,
but finally, I realized I do not have to care so much about language performance. The bottleneck of webscraping is
not on the performance of the language. The bottleneck is on
network I/O, HTML parsing and multitasking. All I have to do
is make sure the program works and find the bottleneck.
Finally, I chose Python rather than Lua. There is so
many excellent Python modules; I have no reason to build my
own.
According to my experience about webscraping, I chose
Twisted for network I/O and lxml for html parsing as the backend
of my webscraping program. I have wrote an article for an introduction to this technology.
The best choice to grab data from websites: Python + Twisted + lxml
Hope this is helpful.
I think the answer about it being a “marketing” thing is probably correct, along with the lack of a large set of libraries to choose from. I would like to point out another case of this: Ruby. Ruby is meant to be a general purpose scripting language. The problem is that since Ruby on Rails has risen to be so popular, it is becoming hard to find something that is unrelated to Rails. I’m afraid Lua will suffer this as well, being popular because of a few major things using it, but never able to break free of that stigma.
Lua has fewer libraries than Python. But be sure to have a look at LuaForge. It has a lot of interesting libs, like LuaCURL, wxLua or getopt.
Then, visit LuaRocks, the package management system for Lua. With it, you can search and install most mature Lua modules with dependencies. It feels like RubyGems or aptitude.
The site lua-users.org has a lot of interesting resources too, like tutorials or the Lua Wiki.
What I like about Lua is not its speed, it’s its minimal core language, flexibility and extensibility.
That said, I would probably use Python for the tasks you mentionned because of the larger community doing such things in Python.
This is a sociological question, not a programming question.
I use Lua for general-purpose scripting almost exclusively. But I had to write a few hundred lines of code so that Lua would play better with the shell. This included such tricks as
- Quoting a string so it is seen as one word by the shell
- Writing a function to capture the output of a command as in shell $(command)
- Writing a function to crawl the filesystem using the Lua posix library and expand shell globbing patterns
(For those who may be interested, I’ve left the code in my Lua drop box, which also contains some other stuff. The interesting stuff is probably in osutil in os.quote, os.runf, os.capture, and maybe os.execve. The globbing is in posixutil.lua. They both use Luiz Henrique de Figuereido’s Lua Posix library.)
To me, the extra effort is worth it because I can deal with simple syntax and great data structures. To others, a more direct connection with the shell might be preferred.
Lack of standard library. Period. Even listing all the files in a directory require a non-standard module.
There are good reasons for that (keeping strict ANSI portability, not requiring POSIX) but the result is that, for general programming, I prefer Python.
There has been a recent push to create a batteries included installation for Lua on Windows. The result can be found at the Lua for Windows project at LuaForge. It includes the interpreter and a large collection of extra modules allowing useful scripts and applications to be written and used out of the box.
I know that various Linux distros are including Lua and some modules now, and more to come.
There are also a couple of proposed module libraries under discussion in the mailing list, but the community hasn’t yet settled on one as the “official” mechanism.
I use Lua both as a scripting language and as the “main” loop of my typical application, supported by one or more DLLs containing code that was better implemented in C, or wrapping existing libraries or API functions that are needed by a particular project. Used with a GUI toolkit such as IUP or wxLua (a Lua binding for wxWindows), Lua makes writing small to mid-sized GUI applications quite pleasant.
In order for Lua to be easy to embed it has to have few dependencies and be small. That makes it poorly suited as a general purpose scripting language. Because using it as a general purpose script language would require a lot of standard libraries. But if Lua had a lot of standard libraries it would be harder to embed (due to dependencies and memory footprint.)
Lua is used in LuaTeX, a TeX extension, as an embedded language, and has gained popularity rapidly among TeX developers because of that. It is used as a scripting language for some utilities in the TeX Live distribution, be it only because now there is a luatex binary, available on all platforms, that can also be used as a Lua interpreter (with some vital modules added – slnunicode, luafilesystem, etc.) That’s very important for Windows installations, that relied on additional Unix scripting tools earlier (ActivePerl, etc.) The ConTeXt macro language uses Lua scripts extensively nowadays.
That’s admittedly a very special field 🙂 But completely unrelated to games!