Planet Python

Python Diary: DVD Collection source code now available

Sun, 19 May 2013 17:41:21 +0000

For those who were wanting a copy of the DVD Collection software made in Python, I have now open sourced it and it is live on BitBucket!

DVD Collection software

Python Diary: Python Script to encode Django templates

Sun, 19 May 2013 17:41:21 +0000

Do you need to display raw Django template code in your Django 1.4 project? Look no further than this script! It's rather crude, but gets the job done. I haven't yet updated a few Django websites to Django 1.5, which has a new template tag to do this for you, so I created this script to use in legacy Django sites, and it works like a charm!

#!/usr/bin/python

import sys

try:
  filename = sys.argv[1]
except IndexError:
  print "This command needs exactly 1 parameter!"
  sys.exit()

data = open(filename, 'r').read()
data = data.replace('{%', '{! templatetag openblock !}').replace('%}', '{! templatetag closeblock !}')
data = data.replace('{{', '{% templatetag openvariable %}').replace('}}', '{% templatetag closevariable %}')

print data.replace('{!', '{%').replace('!}', '%}')

You should Pygments to highlight the syntax like I do on this blog of course. If you are using Django 1.5 or greater, you should use the verbatim template tag over this.

Python 4 Kids: Comprehending Lists and Tuples

Sun, 19 May 2013 09:11:52 +0000

SUPERIMPOSED CAPTION: ‘SUBURBAN LOUNGE NEAR ESHER’
Elderly couple, Mr A and Mrs B are staring through french windows at a cat that is sitting in the middle of their lawn motionless and facing away from them. A car is heard drawing up.

The last couple of tutorials have been a bit heavy, so this week’s tutorials are going to balance that out by being a little light. We’re having a look at a couple of different aspects of the Python language.

List Comprehensions
The first is comprehensions:

>>> a_list = [1,2,3,4,5]
>>> b_list= [element*2 for element in a_list]
>>> b_list
[2, 4, 6, 8, 10]

In this example, we have automatically generated a new list b_list from an existing list a_list by using a list comprehension. The structure of the comprehension is, I hope comprehensible. If not, it is a little hard to explain in steps. The comprehension above is equivalent to:

>>> b_list = []
>>> for element in a_list:
...    b_list.append(element*2)
... 
>>> b_list
[2, 4, 6, 8, 10]

We could call the variable element anything we liked – it is just another variable:

 
>>> b_list = [baloney*2 for baloney in a_list]
>>> b_list
[2, 4, 6, 8, 10]
>>>

You can also add conditions. Let’s say you only wanted the even elements in a_list:

 
>>> b_list = [element for element in a_list if element%2 ==0] #ie remainder is 0 when dividing by two
>>> b_list
[2, 4]
>>>

The condition here is if element%2 ==0, but you can substitute other conditions (as long as they resolve to either True or False). You can have a comprehension which relies on multiple variables (this example is from the Python documentation):

 
>>> [(x, y) for x in [1,2,3] for y in [3,1,4] if x != y]
[(1, 3), (1, 4), (2, 3), (2, 1), (2, 4), (3, 1), (3, 4)]

Unwinding this is a little tricky – for each element in [1,2,3] it runs through each element in [3,1,4] (ie 9 comparisons in all) and appends the tuple if the elements are not equal.

List comprehensions provide an easy shorthand for constructing lists. Moreover, they are often more readable than writing out the for loops explicitly.

Tuples
The second topic is tuples.
The other thing we’re going to look at is tuples (pronounced, variously, “tupp-lls”, “two-pls” and, to some people, “tyou-pls” – my preference is tupp-lls, rhymes with couples). Tuples are a little like lists, except that they are immutable. That is, once they are made they cannot be changed. Tuples are made by putting the elements in round braces:

 
>>> my_tuple = (1,2,3)
>>> my_tuple
(1, 2, 3)
>>> my_list = [1,2,3]
>>> my_list
[1, 2, 3]

Elements of the tuple are referenced in the same way as you’d reference a list but, unlike lists, these elements cannot be changed (tuples are “immutable”):

 
>>>my_tuple[0]                                                                                                       1                                                                                                                    >>> my_tuple[0]=2
Traceback (most recent call last):                                         
  File "<stdin>", line 1, in <module>                                      
TypeError: 'tuple' object does not support item assignment

The TypeError is telling us that you can’t change the elements of a tuple. To make a tuple with a single element is a little difficult because the interpreter could just interpret the parentheses as determining order of operation (think (1+2)*3). Instead, we put a comma after the single element to indicate that we are creating a tuple:

 
>>> my_tuple = (1)  # no comma, so Python just thinks it's a number
>>> my_tuple  
1             
>>> my_tuple[0]  # just a number, so has no elements
Traceback (most recent call last):                                                                           
  File "<stdin>", line 1, in <module> 
TypeError: 'int' object has no attribute '__getitem__'
>>> my_tuple= (1,)
>>> my_tuple
(1,)                                                                                                                 
>>>my_tuple[0]                                                                                                       
1

So, if a tuple does what a list can do, only less, why bother with a tuple? Why not use a list? Well, tuples are easier for the Python interpreter to deal with and therefore might end up being faster. Tuples might also indicate that you have an ordered list where the ordering has some meaning (for example, a date tuple might store year, month and day (in that order) in tuple). The fact that you’re using a tuple then flags that each of the entries has a distinct meaning or use and that their order is significant. Another pragmatic reason to use a tuple is when you have data which you know shouldn’t change (like a constant). So, if you accidentally try to change one of the tuple’s entries, you will have an error thrown.

Finally, since tuples are immutable they can be used as keys in dictionaries, unlike lists:

 
>>> my_dict={}
>>> my_tuple=("A Name",23,"A location")
>>> my_list= list(my_tuple)
>>> my_dict[my_tuple]="Client 1"
>>> my_dict[my_list]="Client 2"
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: unhashable type: 'list'

Holger Krekel: PEP438 is live: speed up python package installs now!

Sun, 19 May 2013 08:49:44 +0000

My “speed up pypi installs” PEP438 has been accepted and transition phase 1 is live: as a package maintainer you can speed up the installation for your packages for all your users now, with the click of a button: Login to https://pypi.python.org and then go to urls for each of your packages, and specify that all release files are hosted from pypi.python.org. Or add explicit download urls with an MD5. Tools such as pip or easy_install will thus avoid any slow crawling of third party sites.

Many thanks to Carl Meyer who helped me write the PEP, and Donald Stufft for implementing most of it, and Richard Jones who accepted it today! And thanks also to the distutils-sig discussion participants, in particular Phillip Eby and Marc-Andre Lemburg.

Mathieu Virbel: Kivy 1.7.0 is out

Sun, 19 May 2013 07:48:23 +0000

If you missed it, Kivy 1.7.0 is out last Monday. It’s quite a big release that include some big fixes we did during the :

Android Emulator is now supported
ScrollView has been re-factored with new physics calculations. To be exact, the scrollview doesn’t calculate anything now, it just pass the touch position to an Scroll effect. This class calculate the movement’s velocity, and the over-scroll’s distance (means how far you scrolled out of the bounds). Then, we implemented 2 visual effects that use the over-scroll to make the scrollview act as a Damped spring , and/or to fade out the scrollview if you over-scroll too much.
Garden: a separated organization for centralize user’s widgets and addons. You can create garden packages very easily, and import it in the source code with just “kivy.garden.packagename“. Few garden packages are already available!
And more, Check the announcement

If you like my work, tip me!

Twisted Matrix Labs: Migration Report

Sun, 19 May 2013 00:45:55 +0000

I have completed the migration scripts for deploying the services currently running on cube. They have been run against our new machine, dornkirk which is currently running with a snapshot of data.

It can currently be accessed by putting

66.35.39.66     twistedmatrix.com speed.twistedmatrix.com

in /etc/hosts. Please tests it, and verify that things appear to be working, but be aware that any changes will be lost, when the transition occurs.

At some point Monday or Tuesday, there will be some downtime for mail and the mailing lists, as mail-service is migrated to the new machine. For those that have accounts on cube, your data will be copied to the new machine at this point.

On Wednesday, at about 10 MDT (16 UTC), there will be downtime of all twisted services, as live data is transfered over. This may last up-to a couple of hours.

This work is made possible by the sponsorship of individuals and organizations which have donated to the Twisted project, part of the Software Freedom Conservancy, a not-for-profit organization that helps promote, improve, and develop open source software. Thanks!

Dariusz Suchojad: Zato 1.0. The next generation ESB and application server. Open-source. In Python.

Sat, 18 May 2013 22:58:51 +0000

(This is a re-post of what I sent to python-announce@ but with several screenshots attached)

I’m very happy to announce the first release of Zato, the next generation ESB and application server, available under a commercial-friendly open-source LGPL license.

https://zato.io

What can you expect out of the box?

HTTP, JSON, SOAP, Redis, AMQP, JMS WebSphere MQ, ZeroMQ, FTP, SQL, hot-deployment, job scheduling, statistics, high-availability load balancing and more
Incredible productivity with Python
Painless rollouts with less downtime
Slick web admin GUI, CLI and API
Awesome documentation (several hundred A4 pages)
24×7 commercial support and training

Project’s site: https://zato.io
Download: https://zato.io/download/zato-1.0.tar.bz2
Support: https://zato.io/support
Docs: https://zato.io/docs
Architecture: https://zato.io/docs/architecture/overview.html
Tutorial: https://zato.io/docs/tutorial/01.html
GitHub: https://github.com/zatosource
Mailing list: https://mailman-mail5.webfaction.com/listinfo/zato-discuss
IRC: irc://irc.freenode.net/zato
Twitter: https://twitter.com/zatosource
LinkedIn: https://www.linkedin.com/groups?gid=5015554
Diversity statement: https://zato.io/docs/project/diversity.html

Spread the news and enjoy

Cheers!

@fourthrealm

kdev-python: kdev-python 1.5.1 released

Sat, 18 May 2013 14:43:11 +0000

kdev-python provides Python language support for the KDevelop integrated development environment. For more information, please look at the 1.4 and 1.5 release announcements, and especially make sure to watch this short demonstration video!

The 1.5.1 release fixes a few issues with 1.5.0. Especially, it

fixes a crash bug which could happen when PEP8 checking was enabled
fixes capitalization in the coding line completion
displays the correct plugin version in the About dialog

This is a rather minor update and I recommend it to everyone running 1.5.

There's one thing I forgot to announce in the 1.5 release: PEP8 checking support (which was actually a good thing, since it was rather crashy until now ;)).

PEP8 checking suppot in kdev-python 1.5.1

It will run the PEP8 style checker whenever an open file is modified and display all errors inline (if you have inline error display enabled) and in the Problems toolview.
By default it is disabled though, since it displays a lot of errors for people who don't follow PEP8, so if you want to use it, go to Settings -- Configure KDevelop -- PEP8 style checking (the change will only apply to newly opened or modified files).

If there's any issues, please make sure to let me know on the bug tracker.

The tarball can be found here, sha256 checksum is d9b68bd2dd9361961e264254d2acfebc9ce0ea4b47ea2689d2f01a3ed81f7c47

Vern Ceder: Another Quick Python Deal

Sat, 18 May 2013 13:49:55 +0000

Just a quick note that the Quick Python Book, 2nd edition will be Manning Publications deal of the day Saturday, May 18.

Here’s the official scoop:

Deal of the Day : Half off my book The Quick Python Book, Second Edition. Use code dotd0518au at http://www.manning.com/ceder/.

Also on the same code is Hello! Python (http://www.manning.com/briggs/) and Extending jQuery (http://www.manning.com/wood/)

And for anyone in Europe and the Americas, you should know that the deals usually run until past the end of day everywhere in the world. So if you don’t get it done on Saturday, the code might still work on Sunday morning. Just sayin’…

Filed under: Python

Kay Hayen: Nuitka Release 0.4.3

Sat, 18 May 2013 10:17:00 +0000

This is to inform you about the new stable release of Nuitka. Please see the page "What is Nuitka?" for clarification of what it is now and what it wants to be.

This release expands the reach of Nuitka substantially, as new platforms and compilers are now supported. A lot of polish has been applied. Under the hood there is the continued and in-progress effort to implement SSA form in Nuitka.

New Features

Support for new compiler: Microsoft Visual C++.

You can now use Visual Studio 2008 or Visual Studio 2010 for compiling under Windows.
Support for NetBSD.

Nuitka works for at least NetBSD 6.0, older versions may or may not work. This required fixing bugs in the generic "fibers" implementation.
Support for Python3 under Windows too.

Nuitka uses Scons to build the generated C++ files. Unfortunately it requires Python2 to execute, which is not readily available to call from Python3. It now guesses the default installation paths of CPython 2.7 or CPython 2.6 and it will use it for running Scons instead. You have to install it to" C:Python26" or "C:Python27" for Nuitka to be able to find it.
Enhanced Python 3.3 compatibility.

The support the newest version of Python has been extended, improving compatibility for many minor corner cases.
Added warning when a user compiles a module and executes it immediately when that references __name__.

Because very likely the intention was to create an executable. And esp. if there is code like this:
```
if __name__ == "__main__":
   main()
```
In module mode, Nuitka will optimize it away, and nothing will happen on execution. This is because the command
```
nuitka --execute module
```
is behavioral more like

python -c "import module"

and that was a trap for new users.
All Linux architectures are now supported. Due to changes in how evaluation order is enforced, we don't have to implement for specific architectures anymore.

Bug Fixes

Dictionary creation was not fully compatible.

As revealed by using Nuitka with CPython3.3, the order in which dictionaries are to be populated needs to be reversed, i.e. CPython adds the last item first. We didn't observe this before, and it's likely the new dictionary implementation that finds it.

Given that hash randomization makes dictionaries item order undetermined anyway, this is more an issue of testing.
Evaluation order for arguments of calls was not effectively enforced. It is now done in a standards compliant and therefore fully portable way. The compilers and platforms so far supported were not affected, but the newly supported Visual Studio C++ compiler was.
Using a __future__ import inside a function was giving an assertion, instead of the proper syntax error.
Python3: Do not set the attributes sys.exc_type, sys.exc_value, sys.exc_traceback.
Python3: Annotations of function worked only as long as their definition was not referring to local variables.

New Optimization

Calls with no positional arguments are now using the faster call methods.

The generated C++ code was using the () constant at call site, when doing calls that use no positional arguments, which is of course useless.
For Windows now uses OS "Fibers" for Nuitka "Fibers".

Using threads for fibers was causing only overhead and with this API, MSVC had less issues too.

Organizational

Accepting Donations via Paypal, please support funding travels, website, etc.
The User Manual has been updated with new content. We now do support Visual Studio, documented the required LLVM version for clang, Win64 and modules may include modules too, etc. Lots of information was no longer accurate and has been updated.
The Changelog has been improved for consistency, wordings, and styles.
Nuitka is now available on the social code platforms as well
Removed "clean-up.sh", which is practically useless, as tests now clean up after themselves reasonably, and with git clean -dfx working better.
Removed "create-environment.sh" script, which was only setting the PATH variable, which is not necessary.
Added check-with-pylint --emacs option to make output its work with Emacs compilation mode, to allow easier fixing of warnings from PyLint.
Documentation is formatted for 80 columns now, source code will gradually aim at it too. So far 90 columns were used, and up to 100 tolerated.

Cleanups

Removed useless manifest and resource file creation under Windows.

Turns out this is no longer needed at all. Either CPython, MinGW, or Windows improved to no longer need it.
PyLint massive cleanups and annotations bringing down the number of warnings by a lot.
Avoid use of strings and built-ins as run time pre-computed constants that are not needed for specific Python versions, or Nuitka modes.
Do not track needed tuple, list, and dict creation code variants in context, but e.g. in nuitka.codegen.TupleCodes module instead.
Introduced an "internal" module to host the complex call helper functions, instead of just adding it to any module that first uses it.

New Tests

Added basic tests for order evaluation, where there currently were None.
Added support for "2to3" execution under Windows too, so we can run tests for Python3 installations too.

Summary

The release is clearly major step ahead. The new platform support triggered a whole range of improvements, and means this is truly complete now.

Also there is very much polish in this release, reducing the number of warnings, updated documentation, the only thing really missing is visible progress with optimization.

Vasudev Ram: Python's inspect module is powerful

Fri, 17 May 2013 19:44:22 +0000

27.13. inspect — Inspect live objects — Python v2.7.5 documentation

The inspect module comes as part of the standard library of Python. It allows you to inspect live objects at run time by using its methods.

Here is an example:

import inspect
class Bar( object):
    def foo( self ):
        print "in Bar.foo 4"
        self .a = 1
        self .di = { 'b' : 2, 'c' : 3 }
        self .li = [ 4, 5, 6 ]

bar = Bar()
bar.foo()
for member in inspect.getmembers(bar):
print member

Running the above code gives this as (partial) output:

in Bar.foo 4

('__dict__', {'a': 1, 'li': [4, 5, 6], 'di': {'c': 3, 'b': 2}})
('__doc__', None)
('a', 1)
('di', {'c': 3, 'b': 2})
('foo', bound method Bar.foo of __main__.Bar object at 0x4035bfac)
('li', [4, 5, 6])

This shows both the bound methods and the member variables of the instance bar.

The inspect module can do a lot of other things too. Check the docs for it, linked at the top of this post.

You can also modify and run the above code snippet at this codepad.org URL:

http://codepad.org/dMLmkhQ6

It may not last there for too long, though, since they probably delete pastes to make room for new ones.

- Vasudev Ram
dancingbison.com

Joe Abbate: Pyrseas contributions solicited

Fri, 17 May 2013 18:48:31 +0000

Do you use PostgreSQL and truly believe it’s “the world’s most advanced open source database” and that its upcoming 9.3 release will make it even more awesome?

Do you also use Python and believe it’s “an easy to learn, powerful programming language” with “elegant syntax” that makes it an ideal language for developing applications and tools around PostgreSQL, such as Pyrseas?

Then we could use your help. For starters, we want to add support for the MATERIALIZED VIEWs and EVENT TRIGGERs coming up in PG 9.3.

We have also been requested to add the capability to load and maintain “static data” (relatively small, unchanging tables) as part of yamltodb, so that it can be integrated more easily into database version control workflows.

And for the next release, Pyrseas 0.7, we’d like to include the first version of the database augmentation tool which will support declarative implementation of business logic in the database–starting off with audit trail columns. Some work has been done on this already, but it needs integration with the current code and tests.

Or perhaps coding is not your forte, but you’re really good at explaining and documenting technical “stuff”. Then you could give us a hand with revamping the docs, maybe writing a tutorial so that users have a smooth ride using our tools.

Or maybe you have your own ideas as to how improve the PostgreSQL version control experience. We’d love to hear those too.

If you’d like to help, you can fork the code on GitHub, join the mailing list and introduce yourself, or leave a comment below.

Filed under: PostgreSQL, Python, Version control

Montreal Python User Group: PyCon 2014 Call For Launch Day Sponsors

Fri, 17 May 2013 14:32:32 +0000

It’s that time again! Planning for PyCon 2014 is well underway, and we’re currently preparing for the launch of our new site. With the launch comes a unique opportunity: Is your organization interested in being a launch day sponsor?

PyCon 2013 launched on July 9, 2012 with 21 sponsors pledging support, leading the charge that drew over 150 organizations to pitch in to the biggest and best Python conference yet. We’re planning a similar go-live date for the 2014 site, and we’re building up our cadre of supporters for the April 9-17 conference taking place in Montreal, Quebec, Canada.

Your organization’s support enables PyCon to do the awesome things. 2013 introduced a number of new events suggested by the community, so we’d like to keep doing those things and more. For example, the inaugural Young Coders tutorial was such a hit that there are already plans around the world for user groups to replicate it, and we’re looking forward to doing a PyCon 2014 rendition. Programs like Financial Aid, which saw its budget increased and then quickly doubled to reach $100,000 USD, are greatly enhanced by the giving of sponsors.

Along with benefiting the community, sponsorship of PyCon brings many benefits to its supporters. We hear it year after year that there is no better place to hire Python developers than at PyCon. We offer sponsors a place on our site to promote available positions, and we run a job fair on-site that has been a huge success. The Expo Hall is a great place to market your latest projects and to network with 2,000 eager Python developers. The value is unparalleled in the conference scene, especially after considering our flexibility to work with each and every organization. We even offer a 50% discount to organizations under 25 people. See https://us.pycon.org/2013/sponsors/whysponsor/ for more thoughts.

While we’re still finalizing the sponsorship prospectus, it will be very similar to the one we used in 2013 at https://us.pycon.org/2013/sponsors/prospectus/. We’ll share the details as soon as we complete them, and any questions can be forwarded to our Sponsorship Chair, Jesse Noller.

For 2014, PyCon will have a maximum capacity of 2,000 attendees. We’ve sold out the last two conferences and we’re expecting a third, so mark your calendars for April 9-17, 2014. Other dates to remember are our Call for Proposals in July, and we’re looking forward to opening registration in September. We’re planning for the conference schedule to be laid out in December, just in time for the holidays.

If you don’t have a passport, don’t forget that entering Canada requires one. US residents should see http://travel.state.gov/passport/ for details.

PyCon: PyCon 2014 Begins! Call For Launch Day Sponsors

Fri, 17 May 2013 11:34:54 +0000

French translation: http://montrealpython.org/fr/2013/05/pycon-day-sponsors/

It's that time again! Planning for PyCon 2014 is well underway, and we're currently preparing for the launch of our new site. With the launch comes a unique opportunity: Is your organization interested in being a launch day sponsor?

PyCon 2013 launched on July 9, 2012 with 21 sponsors pledging support, leading the charge that drew over 150 organizations to pitch in to the biggest and best Python conference yet. We're planning a similar go-live date for the 2014 site, and we're building up our cadre of supporters for the April 9-17 conference taking place in Montreal, Quebec, Canada.

Your organization's support enables PyCon to do the awesome things that it does. 2013 introduced a number of new events that we've heard great feedback on, so we'd like to keep doing those things and more. For example, the inaugural Young Coders tutorial was such a hit that there are already plans around the world for user groups to replicate it, and we're looking forward to doing a PyCon 2014 rendition. Programs like Financial Aid, which saw its budget increased and then quickly doubled to reach $100,000 USD, are greatly enhanced by the giving of sponsors.

Along with benefiting the community, sponsorship of PyCon brings many benefits to its supporters. We hear it year after year that there is no better place to hire Python developers than at PyCon. We offer sponsors a place on our site to promote their open positions, and we run a job fair on-site that has been a huge success. The Expo Hall is a great place to market your latest projects and to network with 2,000 eager Python developers. The value is unparalleled in the conference scene, especially after considering our flexibility to work with each and every organization. We even offer a 50% discount to organizations under 25 people. See https://us.pycon.org/2013/sponsors/whysponsor/ for more thoughts.

While we're still finalizing the sponsorship prospectus, it will be very similar to the one we used in 2013 at https://us.pycon.org/2013/sponsors/prospectus/. We'll share the details as soon as we complete them, and any questions can be forwarded to our Sponsorship Chair, Jesse Noller.

For 2014, PyCon will have a maximum capacity of 2,000 attendees. We've sold out the last two conferences and we're expecting a third, so mark your calendars for April 9-17, 2014. Other dates to remember are our Call for Proposals in July, and we're looking forward to opening registration in September. We're planning for the conference schedule to be laid out in December, just in time for the holidays.

If you don't have a passport, don't forget that Canada requires one. US residents should see http://travel.state.gov/passport/ for details.

Reinout van Rees: Principle philosophy - Swift

Fri, 17 May 2013 09:23:00 +0000

Principle philosophy: a way to discuss our rules and beliefs that govern our actions. He tells it from his personal experience.

His parents wanted to raise him as a good person. So they thought him good principles (like don't be a quitter, don't steal, etc). This is quite black/white though. We are all more gray/gray.

What about the question "how can I be a good programmer"? Programmers use logic, which sounds black/white again: write tests, don't repeat yourself. Sigh.

Talking about things like this is impossible without Immanuel Kant. He differentiates between reason and instinct. If "be happy" were our life goal, we'd just follow our instincts. So what is reason for, then, apart for doing good? Reason has to do with moral. There are three ways of looking at "doing good":

Duty. Good things can come from duty. Duty can also lead to non-good things, though. Hm, so this is not it.
Make a difference between the goal and the outcome. The outcome might be bad even though the goal could be worthy.
Universal lawfullness. Only do something if you know that everybody thinks it is a good idea.

Does this help with a question like "is testing good"?

Gandhi said that a man is the sum of his actions.

In a sense we are the sum of our experiences. So increase the amount of experience that you have. Either have the experiences yourself, or share them like on this conference. Everything looks different from the trenches: learn from eachother.

Some lessons he learned from a little baseball league experience (where he sucked) as a kid:

Swing for the fences. Aim for a home run. It allows you take great risks (because you have great goals). It motivates you.
Set reasonable goals, too. Incremental intermediate goals. Those intermediate goals help you progress.
You suck... and that's totally OK. You're not good at everything. It gives you a different perspective. And you can still give it your best. Also to that almost-unused old project that you get a bug report for.

Some take-aways:

Build a strong foundation of principles.
One size doesn't fit all
Learn from your experiences and share them.
Build a great network.
Ask all the right questions.

PyPy Development: PyPy 2.0.1 - Bohr Smørrebrød

Thu, 16 May 2013 20:10:08 +0000

We're pleased to announce PyPy 2.0.1. This is a stable bugfix release over 2.0. You can download it here:

http://pypy.org/download.html

The fixes are mainly about fatal errors or crashes in our stdlib. See below for more details.

What is PyPy?

PyPy is a very compliant Python interpreter, almost a drop-in replacement for CPython 2.7. It's fast (pypy 2.0 and cpython 2.7.3 performance comparison) due to its integrated tracing JIT compiler.

This release supports x86 machines running Linux 32/64, Mac OS X 64 or Windows 32. Support for ARM is progressing but not bug-free yet.

Highlights

fix an occasional crash in the JIT that ends in RPython Fatal error: NotImplementedError.
id(x) is now always a positive number (except on int/float/long/complex). This fixes an issue in _sqlite.py (mostly for 32-bit Linux).
fix crashes of callback-from-C-functions (with cffi) when used together with Stackless features, on asmgcc (i.e. Linux only). Now gevent should work better.
work around an eventlet issue with socket._decref_socketios().

Cheers, arigo et. al. for the PyPy team

Reinout van Rees: The web of stuff - Zack Voase

Thu, 16 May 2013 15:47:00 +0000

A plane flew over (noisily) at the start of his presentation. He put our work in perspective by saying that that was a 80 ton plane and that we're just building websites :-)

Possibilities

Computers used to take up whole rooms, now you have a smartphone. Big data is really big data now. Moore's lawworks both ways, though, so you have really small computers now. An arduino for instance.

He often makes comparison to the human body. All over our body, sensors give off signals that go into the central nervous system. The brain processes it and gives signals back to muscles if necessary. Sensing, feedback, understanding, reaction.

Stuff can talk to the cloud. Like a sensor in your body talks to your mind, stuff can treat the cloud as a brain. The cloud is what allows small tools to be smart.

Stuff does often need a human to interact with it. Like a smartphone. There's all sorts of people thinking about how to "liberate the computers from their human overlords". Why cannot computers sense and act on their own account?

So how do you bridge the gap betwen sensing and acting of stuff? How do you use Django for it? There's a lot available online about sensing and about acting, but not the communication in between.

The communication medium itself is a bit of a problem. You don't want to have a telephone data contract for every single small piece of stuff. A physical connection isn't always handy either.

His preferred communication medium is Twilio for sending SMSs. The stuff has low memory, so the message length limit is fine.

He showed a demo with a card reader that read his London transport card and sended an SMS to his Django site. The card reader was a combination between an arduino, a 'shield' sms sender and an RFID reader. The django app then submits it to foursquare. (The last part didn't work, probably due to a local foursquare problem, but the django app did have all the data he send from his card reader). Nice.

SUCCESS: after the lightning talks he did it again and now it worked!

Personal development

He had never done any hardware work until four months ago. No compiling for arduino. It sounded a bit scary to him.

It is normal, if you start as a beginner, you're slowly getting better if you keep at something. Then you automatically learn more and thus learn that there's a lot you don't know. That's the dip in the middle. Those are the people we need to keep on board so that they push through to the expert stage.

When you're in the middle, you know how bad you are (or how good you aren't yet). That's the risky phase were people quit.

Likewise documentation. Tutorials are useful for beginners. Reference material is useful for experts. There's not a lot in the middle and you're bound to be a bit frustrated in that stage.

So if you're going to start experimenting with electronics, you're bound to hit a wall, for instance when calculating complex electronic schemas. Push on anyway: the first time you make a phone call with your own device is totally worth it.

Two books he recommends to get you started:

Getting started with Arduino.
The art of electronics.

Reinout van Rees: Growing open source seeds - Kenneth Reitz

Thu, 16 May 2013 12:02:00 +0000

He shows us three kinds of (more or less) open source projects.

Type 1: public source

Once upon a time there was an "open source project" called the facebook SDK. Basically it just stopped working one day and nobody could help, despite offers for help on the issue tracker. Hacker news got wind of it and it was on the front page for a while. Facebook's reaction? Disabling the issue tracker... (Later on they fixed it).

That's not open source, that's public source. Often it is abandoned due to loack of interest, change of focus or so. The motivation for having it as open source simply is not clear.

Type 2: shared investment

A different example: gittip. They aim to be the world's first open company. There's a github issue for everything, even the company name. Major decisions are voted for on github. The code is open source, of course. All interviews with journalists are filmed and live-streamed. And all otherwise-often-backdoor-cooperation-agreements are fully open.

Projects like gittip are shared investment projects. Shared ownership, extreme transparency. There is very little questioning of motivations. The motivation is clear and public. There's a documented process for new contributers. The advantage? It is low risk. There's a high bus factor.

Type 3: dictatorship project

Kenneth is the author of requests. An open source project, very succesful. But all the decisions are made by Kenneth.

That's really more of an dictatorship project. A totalitarian BDFL that owns everything. The dictator is responsible for all decisions. Requests' values lie in its extreme opinions. If he'd involve more people, the value would be dilluted. There are drawbacks. A low bus factor. High risk of burnout: Kenneth is the single point of failure.

Lessons learned

Be cordial or be on your way. As a user, you need to keep all your interactions with the maintainer as respectful as possible. The maintainer put a lot of work in it and they don't owe you any of their time.

As a maintainer, you also must be cordial. Be thankful to all contributions. Feedback is the liveblood of your project, even the negative. You'll need to ignore non-constructive comments. Be careful with the words you choose, sometimes contributors take what you say VERY personally. You might have to educate your users. And: a bit of kindness goes a long way.
Sustainability is almost the biggest challenge. Don't burn out. Try to get others to help.

He quotes Wes Beary: "open source provides a unique opportunity for the trifecta of purpose, mastery and autonomy". Pay equal attention to all of these three. Learn to do less, focus more on your purpose, for instance.
Learn to say no. People ask for crazy features. Or they submit quite sane pull requests that, if you allow them all in, makes your project slow and unfocused. Kenneth wants as few lines of codes in his project. Negative diffs are the best diffs!
Open source makes the world a better place. Don't make it complicated!

Tryton News: Last maintenance releases for series 1.8

Thu, 16 May 2013 12:00:00 +0000

Reinout van Rees: Advanced Python through Django: metaclasses - Peter Inglesby

Thu, 16 May 2013 11:33:00 +0000

Metaclasses are a handy feature of Python and Django makes good use of them.

When you create certain kinds of classes in Django, a metaclass will do something to the class before it is created. For forms, the various attributes of the class are converted into a base_fields dictionary on the class.

Similarly, a subclass of Model also fires up a metaclass that does some registering. A foreignkey to another model adds a relation back on that other model, for instance.

As a recap, a class is something that can be instantiated into an object. It can have an __init__() method that does something upon instantiation. type(your_instance) will return the class.

Did you know that you can create classes dynamically? See for yourself:

>>> name = 'ExampleClass'
>>> bases = (object,)
>>> attrs = {'__init__': lambda self: print('Hello from __init__')}
>>> ExampleClass = type(name, bases, attrs)
>>> example = ExampleClass()
Hello From __init__
>>> type(example)
<class '__console__.ExampleClass'>

So... we can actually control how classes are created! You could create a create_class() method that calls type but that modifies, for instance, the name. Or we could take all the attributes and add them to a base_fields dictionary on the instance. Hey, that's what we saw in the first Django form example!

Now, what is type exactly? It is a class that creates classes.

This also means we can subclass it! The most useful thing to override in our subclass is the __new__() method. The __init__() method creates instances from the class, the __new__() creates classes. So again we can modify the name and/or the attributes.

How do you use it in practice? Normally you'd set a __metaclass__ attribute on a class. This tells python to use that metaclass for creating the class. The same for subclasses. This is how our Django form classes use the metaclass specified in Django's base Form class.

Django uses metaclasses in five places: admin media, models, forms, formfields, form widgets. Grep for metaclass in your local django source code once to get a better feel for how Django uses it.

Note on python 3: it uses a slightly different syntax for specifying metaclasses. So Django 1.5 uses six to support both ways in a single codebase.

Warning: don't overuse metaclasses. They can make code difficult to debug and follow. Use Django as a good example of how to use metaclasses. Django saves you a lot of work by using metaclasses in a few locations.

See https://github.com/inglesp/Metaclasses

Nice way of giving a presentation, btw. Some sort of semi-interactive python prompt. The software is online at https://github.com/inglesp/prescons

Reinout van Rees: Thread profiling in Python - Amjith Ramanujam

Thu, 16 May 2013 11:04:00 +0000

Amjith Ramanujam recently wrote a thread profiler in Python and it was rediculously simple. He works for New Relic, which is all about performance and expecially performance measuring.

Python comes with batteries included, so it also includes a C profiler that works pretty well. But it doesn't work nice for django because the output is so huge. If you use the GUI RunSnakeRun, it is more managable that way.

Additional problem with cProfile: it has about 100% overhead, so you can't run it in production (which they need).

You can do more targeted profiling. For instance in Django. The way a web framework processes requests is normally always in the same way. You can use that during profiling.

There are two important stages: interrupt and inquire.

Interrupt: A statistical profiler looks how often a function is called and by who. For this it needs to interrupt the regular process. You could set an OS-level signal to call your profiler every x miliseconds so that it can do something. It only works in linux, btw. Another way is to create a python background thread that wakes up every x miliseconds. It is cross-platform and mod_wsgi compatible. It is less accurate for CPU tasks and it cannot interrupt C extensions.
Inquire: You can use sys._current_frames() to get yourself the frames ("stack trace") from the current thread. Here you can extract the filename, line number and so of the most interesting frames. He's building a "call tree" structure, mapping how often a function is called, preserving the parent/child relation between functions. The tricky part was visualizing it :-) In the end they did it with d3.js.

There's some 3% overhead in the thread approach, so that's real good. They can switch it on for a type of request and it'll profile the next 100 requests of that type.

Reinout van Rees: Copernicus, the great refactorer - Brandon Rhodes

Thu, 16 May 2013 11:03:00 +0000

Brandon Rhodes mentions Nicolai Kopernik, a famous Polish scientist. He "lifted Earth into Heaven" by his book where he put the sun in the center of our universe instead of Earth. We're no longer at the bottom.

The near-earth environment was pretty well mapped out. 300BC, the size of the spherical earth was already known. Around 100BC the distance to the moon was known.

But what about those planets? They were harder. Stars did move around the sky more or less linearly. But those planets. They seemed to move back and forth a bit. Did they need to have a different model? The sun-centric model was already known, but it didn't catch on. The reason? Medieval science was too emperic: the earth cannot be moving, it seems to stay in place. Throw a ball and it falls down again towards the earth, it doesn't career off into space. The church wasn't totally idiotic by later convicting Galileo: there was just no solid emperical evidence :-)

One of the pieces of evidence that was missing was that there was no observable stellar parralax; visible movement between stars because of earth movement. It was only in 1838 that the instruments were good enough to actually observe parralax. Only then did we figure out how far away the stars were. So it took 200-300 year to get real evidence.

So why did Kopernik trust his theory despite the lack of evidence? Because of beauty. Brandon showed Kopernik's theory as Python code; an algorithm for planet movements. And showed his theory as a code refactoring. The end result was quite simple and nice once you factored out Earth's motion. The code looks nicely ordered.

There's truth in beauty.

Brandon proposes a new term: Kopernican refactoring: making a system simpler by moving something new into the center.

It is so easy to behave according to a rule that's not even really there. You look at a problem in a certain way and you're stuck. Once you put something else in the center, you look at the problem in a different way.

For instance, you can use argparse to generate a help message when you pass --help. You can also do it the other way around: using docopt which generates a parser from your help message! You get a much nicer help message.

So if you're stuck, think of Copernicus and his refactoring.

Reinout van Rees: The imaginative programmer - Zed Shaw

Thu, 16 May 2013 11:03:00 +0000

His goal: teach programmers to be more creative.

He's got a love/hate relationship with creativity. The first part of his talk was impossible to summarize. You'll have to watch the video later on :-)

Artists tell him he's not artistic because he works on developing technical skills.
Guitarists tell him he's not a real guitarist as he doesn't play in a band. And 'cause he builds his own guitars he's a programmer, not a Real Guitarist.
Writers tell him he's not a writer because he writes technical books.
Programmers tell him he's not a programmer because he doesn't work on their project. And by the way, he's a (technical) writer now, so he's not a programmer.

He's not creative enough. Or so the others say. Or he's not acceptable. How to deal with creativity? In a way, you can re-phrase creativity. Programmers are always making something from nothing, right? Isn't that the pinnacle of creativity?

Here are four hypothetical persons:

Technique, no imagination: a stereotypical programmer.
Imagination, no technique: stereotypical biz dude.
No imagination, no technique. Probably doesn't exist.
Both imagination and technique. Zed's goal.

Zed's imaginative programmer process. Everyone has a process (if they're good), here's the one he proposes to help you be more creative:

You start with an idea.
You establish a concept that helps form the idea. It gives your idea clothes.
Research techniques or tools. Do some research or you'll pay for it later on.
Refine the concept through composition. So put a box around the vast world of available possibilities. Just mark which features are inside and outside the concept.
Explore through prototypes. Throw away code or use paper prototypes for instance. This saves you so much time later.
You make it real.

We are programmers, so we should iterate this process.

He tried this process with http://projectzorn.com/, going through all the stages. And he's probably going to work on it

Reinout van Rees: Bleed for speed - Rob Spectre

Thu, 16 May 2013 11:02:00 +0000

He started with a little history lesson. The sea battle of mobile bay. The admiral (Faragut) ordered the ships straight through the minefield (called "torpedoes" at the time). "Damn the torpedoes, full speed ahead". And it worked.

What does this to have to do with Django? Well, "damn the torpedoes, full speed ahead" feels a bit like how rapid prototyping feels afterwards. He's often involved with hackathons. Lot of quick coding in limited time with a lot of people. He learned a lot about his tools that way (and he often used Django).

There's a time to make a distinction between production and prototype. Sometimes it is better to just try something with a prototype. Throw-away code.

Aaargh! Throw-away code?!? We never throw code away. But it is something we must learn. It is good to let go once in a while. Let your code go. It isn't yourself, it is just some code.

The danger is that prototype code is put into action as production code. With some work, this danger can be prevented.

What about Django? Django is the best for prototyping. For rapid prototyping, Django is better than micro-frameworks like Flask that might seem better at first glance. Here are some reasons:

Django was build for rapid prototyping. It originated at a newspaper! 24 hours to build something.
It is flexible. It was build to bend. He can prepare something for the other people programmign with him and get them going and still keep the code in reasonable shape.
Us. The strength of Django is the community that supports it. Stack overflow questions and answers. The django websites. Books like two scoops of Django (see my review). That's not something you have with many other frameworks!

Tip: read especially chapter 2 and 3 of the two scoops book.

One thing he'd add to the book is stuff like fabfiles and makefiles. Handy for rapid prototyping.

Use stuff that's available. For instance Django's staticfiles app for grabbing together all the css/js/png. Whether it is in one directory or split out over multiple apps. It also helps with production.

Also look at brunch for setting up your javascript app's structure. It works well with Django.

Deployment: you need to show your prototype. Heroku is very quick for prototypes. (He mentioned that they have a data center in Europe now in case you need it).

Deployment: use chef. Lots of recipes. You could also use Salt if you're more into Python. Also lots of stuff available. Both take a while to learn, but it is a very good investment.

Configuration management is an extremely useful skill. Do it well.

Tastypie is the quickest way to get a REST api out of your Django. It is the best for rapid prototyping. Another good one is django-rest-framework. It will take a little bit longer to set up, but once done you're working with actual Django views. And django-rest-framework's browseable API is very helpful when you're working with a couple of others

Social auth connectors: everyone makes one and there are way too many half-working ones. He's got two that he can recommend. django-social-auth is very complete. The other is django-allauth for when speed is important for you.

If you don't want to play fair to others during a hackathon: use celery. It is very unfair to use celery, python and Django. The combination with Django is pretty OK to set up. You can do a lot what others cannot do easily. So use it for rapid prototyping. (Regarding setting it up: there are good chef recipes for it).

TEST. Yes, even during a hackathon. He doesn't advocate full test driven development. It is a balance. But the errors that kill you during a hackathon are the errors you make twice. So, for instance, test that all your views simply return a 200 Ok. This already helps prevent a lot of problems.

Look at AngularJS. Even if you don't use the framework itself. Why? It has a great javascript test runner. Good for testing while rapid prototyping.

Amit Saha: New Book: Write Your First Program – C and Python

Thu, 16 May 2013 10:32:25 +0000

I am excited to announce that my book, Write Your First Program is finally available for public consumption. In this day and age, when there are easily accessible programming books galore (in the context of both, price and availability), why write a new book? Through this post, I hope to reach out to the prospective readers and […]