Heavy usage of Python at Google
Question:
Google’s heavy usage of Python, is it just a matter of taste or does it give them a competitive advantage?
Answers:
I have not read the whole article and I don’t know how representative it is but maybe this answers your question: Python at Google.
Ok I read most of it, unfortunately it gives not that much insight but I think it is still an interesting article. Maybe most important:
At Google, python is one of the 3 “official languages” alongside with C++ and Java. Official here means that Googlers are allowed to deploy these languages to production services. (Internally Google people use many technologies including PHP, C#, Ruby and Perl). Python is well suited to the engineering process at Google. The typical project at Google has a small team (3 people) and a short duration (3 months).
Not to forget that Guido van Rossum the creator of Python worked for Google from 2005-2012 😉
I also found this quote but I cannot verify it:
“Python has been an important part of Google since the beginning, and remains so as the system grows and evolved. Today dozens of Google engineers use Python, and we’re looking for more people with skills in this language”
— Peter Norvig, Director of Search Quality at Google
I can’t really give a definitive answer, because by the time I interviewed at Google in 2004 Python was already prominent at Google.
Indeed, there’s one apparently attractive explanation that I can definitely deny: it’s not that Google uses Python because it employs so many prominent Pythonistas — rather, most “prominent Pythonista” googlers joined Google, at least in part, because we knew about Python’s prominence there (possible exceptions include Peter Norvig and Jeremy Hylton, but historically Google’s choice of Python predated even them).
That’s definitely why I first got interested (my publisher let me know about the large amount of copies of my book that Google was purchasing — at first, I thought of it as a good opportunity to sell my freelance consulting service…;-), how I was later able to convince Guido to join us, and, I believe, part of the motivation for such Pythonistas as Greg Stein, Wesley Chun, Fredrik Lundh, Thomas Wouters, Collin Winters, Jeffrey Yasskin, …
It all got started, I believe, because the very earliest Googlers (Sergey, Larry, Craig, …) made a good engineering decision: “Python where we can, C++ where we must” — they used (a subset of) C++ for the parts of the software stack where very low latency and/or tight control of memory were crucial, and Python, allowing more rapid delivery and maintenance of programs, for other parts. At the time, late ’90s, the choice for the latter role was essentially between Python and Perl: other scripting languages were either unripe (I don’t think Ruby was around yet, for example) or had other issues and limitations. Perl was more mature (especially in terms of its ecosystem of available add-ons via CPAN), but Python was deemed to be more readable and maintainable, and interfacing to C++ libraries (via SWIG) was easier.
Java came in later, covering an intermediate niche — and more recently of course Go was developed (though I don’t believe there’s much production work in it yet, as it’s still evolving and maturing). Some specialized languages such as sawzall are also in the mix for very specific tasks, and of course Javascript is very important for browser-side work.
Other languages, including the ones that Greg mentioned back in ’06, are either “kind of accidental” or used for other specific tasks (e.g., Objective C for clients on iPhones or Macs) — e.g., when Google hired its first system administrators, those employees inevitably came with very strong mastery of Perl and Bash, and often used either of those languages to develop some complex internal system; recoding those in Python (for easier deployment and maintainability) has often happened. Others (such as C#) may have been in the mix temporarily due to acquisitions, but, again, recoding in one of the “main Google languages” is always a pretty high priority (in C#’s case, recoding would typically be mostly in Java, as the two languages address similar areas in terms of levels of abstraction).
If you subscribe to Paul Graham’s idea that succinctness == power (i.e. If I can write code in language X in 10 lines to accomplish what you took 100 lines of language Y, my language is more powerful)
These days hardware is cheap, meatware is expensive. I know that I can write a lot more, faster, better AND clearer in Python than I can in other languages, simply because the batteries tend to be included. Sure, it may be slower but are you really going to notice the difference between .05 seconds and .0005 seconds?
In a more real-world example, if Google developers write a program in Python where they never have to worry about garbage collection, and Company X writes a program in C++ and they spend 10% of their time tracking down bugs related to memory allocation, guess who’s done faster – even if the C++ shop manages to write the rest of their code just as fast.
So yes, I’d say it is a competitive advantage to code in Python.
I was reading about Unladen-Swallow, a project that aims to improve the performance of CPython. While browsing through their discussion board I stumbled across this discussion, where Collin Winter (Google employee and Python core developer) answers on the claim that Google employees are being discouraged from using Python for new projects:
Well, simple common sense is going to
limit Python’s applicability when
operating at Google’s scale: it’s not
as fast as Java or C++, threading
sucks, memory usage is higher, etc.
One of the design constraints we face
when designing any new system is,
“what happens when the load goes up
by 10x or 100x? What happens if the
whole planet thinks your new service
is awesome?” Any technology that makes
satisfying that constraint harder —
and I think Python falls into this
category — should be discouraged if
it doesn’t have a very strong case
made in its favor on other merits. You
have to balance Python’s strengths
with its weaknesses: your engineers
may be more productive using Python,
but if they have to work around more
platform-level performance/scaling
limitations as volume increases, do
you come out ahead? etc.
Google’s heavy usage of Python, is it just a matter of taste or does it give them a competitive advantage?
I have not read the whole article and I don’t know how representative it is but maybe this answers your question: Python at Google.
Ok I read most of it, unfortunately it gives not that much insight but I think it is still an interesting article. Maybe most important:
At Google, python is one of the 3 “official languages” alongside with C++ and Java. Official here means that Googlers are allowed to deploy these languages to production services. (Internally Google people use many technologies including PHP, C#, Ruby and Perl). Python is well suited to the engineering process at Google. The typical project at Google has a small team (3 people) and a short duration (3 months).
Not to forget that Guido van Rossum the creator of Python worked for Google from 2005-2012 😉
I also found this quote but I cannot verify it:
“Python has been an important part of Google since the beginning, and remains so as the system grows and evolved. Today dozens of Google engineers use Python, and we’re looking for more people with skills in this language”
— Peter Norvig, Director of Search Quality at Google
I can’t really give a definitive answer, because by the time I interviewed at Google in 2004 Python was already prominent at Google.
Indeed, there’s one apparently attractive explanation that I can definitely deny: it’s not that Google uses Python because it employs so many prominent Pythonistas — rather, most “prominent Pythonista” googlers joined Google, at least in part, because we knew about Python’s prominence there (possible exceptions include Peter Norvig and Jeremy Hylton, but historically Google’s choice of Python predated even them).
That’s definitely why I first got interested (my publisher let me know about the large amount of copies of my book that Google was purchasing — at first, I thought of it as a good opportunity to sell my freelance consulting service…;-), how I was later able to convince Guido to join us, and, I believe, part of the motivation for such Pythonistas as Greg Stein, Wesley Chun, Fredrik Lundh, Thomas Wouters, Collin Winters, Jeffrey Yasskin, …
It all got started, I believe, because the very earliest Googlers (Sergey, Larry, Craig, …) made a good engineering decision: “Python where we can, C++ where we must” — they used (a subset of) C++ for the parts of the software stack where very low latency and/or tight control of memory were crucial, and Python, allowing more rapid delivery and maintenance of programs, for other parts. At the time, late ’90s, the choice for the latter role was essentially between Python and Perl: other scripting languages were either unripe (I don’t think Ruby was around yet, for example) or had other issues and limitations. Perl was more mature (especially in terms of its ecosystem of available add-ons via CPAN), but Python was deemed to be more readable and maintainable, and interfacing to C++ libraries (via SWIG) was easier.
Java came in later, covering an intermediate niche — and more recently of course Go was developed (though I don’t believe there’s much production work in it yet, as it’s still evolving and maturing). Some specialized languages such as sawzall are also in the mix for very specific tasks, and of course Javascript is very important for browser-side work.
Other languages, including the ones that Greg mentioned back in ’06, are either “kind of accidental” or used for other specific tasks (e.g., Objective C for clients on iPhones or Macs) — e.g., when Google hired its first system administrators, those employees inevitably came with very strong mastery of Perl and Bash, and often used either of those languages to develop some complex internal system; recoding those in Python (for easier deployment and maintainability) has often happened. Others (such as C#) may have been in the mix temporarily due to acquisitions, but, again, recoding in one of the “main Google languages” is always a pretty high priority (in C#’s case, recoding would typically be mostly in Java, as the two languages address similar areas in terms of levels of abstraction).
If you subscribe to Paul Graham’s idea that succinctness == power (i.e. If I can write code in language X in 10 lines to accomplish what you took 100 lines of language Y, my language is more powerful)
These days hardware is cheap, meatware is expensive. I know that I can write a lot more, faster, better AND clearer in Python than I can in other languages, simply because the batteries tend to be included. Sure, it may be slower but are you really going to notice the difference between .05 seconds and .0005 seconds?
In a more real-world example, if Google developers write a program in Python where they never have to worry about garbage collection, and Company X writes a program in C++ and they spend 10% of their time tracking down bugs related to memory allocation, guess who’s done faster – even if the C++ shop manages to write the rest of their code just as fast.
So yes, I’d say it is a competitive advantage to code in Python.
I was reading about Unladen-Swallow, a project that aims to improve the performance of CPython. While browsing through their discussion board I stumbled across this discussion, where Collin Winter (Google employee and Python core developer) answers on the claim that Google employees are being discouraged from using Python for new projects:
Well, simple common sense is going to
limit Python’s applicability when
operating at Google’s scale: it’s not
as fast as Java or C++, threading
sucks, memory usage is higher, etc.
One of the design constraints we face
when designing any new system is,
“what happens when the load goes up
by 10x or 100x? What happens if the
whole planet thinks your new service
is awesome?” Any technology that makes
satisfying that constraint harder —
and I think Python falls into this
category — should be discouraged if
it doesn’t have a very strong case
made in its favor on other merits. You
have to balance Python’s strengths
with its weaknesses: your engineers
may be more productive using Python,
but if they have to work around more
platform-level performance/scaling
limitations as volume increases, do
you come out ahead? etc.