Ashley Olsen onto Python. Mary-Kate Still PHP'ing

ashley red
Originally uploaded by benbarren.

As a business person deploying technology written in Python, there is alot of short term pain (like all therapy trying to get to the root cause of the problem ;) Where I underestimated Python is how hard it is to go from PhP to Python - and how that affects development timelines. It seems the allure of GOOG using Python and the mainstreamisation of PhP makes the Python an aspirational programming language. Like Alfa Romeos and Maserati's used to be. Interesting recap of a 'Valley presentation by Greg Stein, who I assume works at (is involved with) google : "At Google, python is one of the 3 "official languages" alongside with C++ and Java. Official here means that Googlers are allowed to deploy these languages to production services. (Internally Google people use many technologies including PHP, C#, Ruby and Perl). Python is well suited to the engineering process at Google. The typical project at Google has a small team (3 people) and a short duration (3 months). After a project is completed developers may move to other projects. Larger projects are sub-divided into 3 month deliverables, and teams get to choose their own language for their project. Engineers are given their 20% time to work on what they want to at Google. Many new ideas spring from this 20% work, and "bottom up" seems to be the mantra at Google. Greg stated that architecture and design were not mandated from the top, but rather the teams working on these projects were given the freedom to suggest and deliver... Internally Google has been using Python 2.2. It has been hard for them to move forward to 2.3 or 2.4 becaue of the large number of machines that the have and they have to have compatibility amoung those machines (I'm assumming this is more of an IT issue, since Python is pretty good at backwards compatibility, but I guess if you deploy 2.4 and start using decorators any machine running 2.2 will choke). Greg said that they will soon try to move to 2.4. Python programmers at Google must follow a strict style guideline (based on PEP8 with 2 spaced indenting). When engineers are first granted commit access to their SCM system, they must pass a style test. All code must pass through two sets of eyes before being checked in. That combined with liberal doses of unittest, pychecker and code coverage eliminates most non-algorithmic issues that might appear in python code... Greg said that really Python is rarely a bottleneck at Google. (With bits going over a network and hitting a database, both of these will impact before python even comes into the picture). As mentioned previously when MS ported their eShops code to .asp, because "Python is interpreted and has to be slow" (not a direct quote, but something like that). When the port to asp was done, the code was in fact slower! When programming in python, one should design for scalability. If you design properly, the you should be able to just throw more machines at the problem and your app should scale fine. Greg's final point on performance was that you can always write in C/C++ and wrap if you need to. This is an attempt to justify the use of python to higher up types who may disapprove of it's use. Greg stated (and this is a direct quote), "People have been saying [wrap C/C++] for 10 years. I've never done this once!".. Someone asked about the use of MySQL in Google. Greg said, "We use it specifically because it is open source". When you buy from a vendor you are subject to the whims of that vendor. Google had an instance where they were using proprietary software and needed a feature to be added. The vendor said no. Google offered to pay for a developer. The vendor said no! Google obviously wasn't happy with this, and open source is one solution. (Obviously not having to pay Oracle a per cpu license for 1000's of machines is probably another reason, but not one that Greg mentioned). There was talk about testing and code quality of python. Greg said that by using code coverage for an interpreted language. Once you have coverage on a line you know it will "compile" much like a C or Java program. So you can get around typos somewhat with that. Someone asked why Google even used Java at all? Greg appeared to bite his tongue and then said that there are a lot of good java programmers out there and Google hires a few of them. On the issue of catching bugs, Greg mentioned that because they are running code on tens of thousands of machines, that they see bugs that appear less frequently A LOT more than most people. They've run into obscure kernel bugs that other people rarely run into. So he said, Google's production code is quite good, because bugs get exposed early and often. There was a question on the GIL (global interpreter lock) in python. Greg said he tried to remove the GIL a few years back, but it was a quick hack and slowed some things down. He said that trying to remove it now would prove very difficult (hopefully PyPy helps here). But Greg's suggestion was to use RPC instead of threading. All in all it was a very good talk, providing interesting insight from a man who has a lot of experience using Python. It was also fun to see where Google is using python inside."