PyCon Ireland 2014

Together with a couple of colleagues, I wrote a short report about PyCon Ireland 2014. Once again the conference was a lot of fun and I'm looking forward to the 2015 edition! :)

Leave a comment

Training at EuroPython 2014 Making your first contribution to OpenStack

OpenStack logo

Last week I ran a 3-hour training on how to get started contributing to OpenStack at EuroPython. The aim was to give a high-level overview of how the contribution process works in the project and guide people through making their first contribution, from account creation to submitting a patch.

Overview

The session starts with an extremely fast overview of OpenStack, geared toward giving the participants an idea of the different components and possible areas for contribution. We then go through creating the accounts, why they're all needed, and how to work with DevStack for the people who have installed it. From there we finally start talking about the contribution process itself, some general points on open-source and OpenStack culture then go through a number of ideas for small tasks suitable for a first contribution. After that it's down to the participants to work on something and prepare a patch. Some people chose to file and triage/confirm bugs. The last part is about making sure the patch matches the community standards, submitting it, and talking about what happens next both to the patch and to the participant as a new member of the community.

Preparing

During the weeks preceding the event, I ran two pilot workshops with small groups (less than 10 people) in my local hackerspace, in preparation for the big one in Berlin. That was absolutely invaluable in terms of making the material more understandable and refining the content for items I didn't think of covering initially (e.g. screen, openrc files) and topics that could use more in-depth explanations (e.g. how to find your first task), timings, and generally getting a feel for what's reasonably achievable within a 3-hour intro workshop.

Delivering

I think it went well, despite some issues at the beginning due to lack of Internet connectivity (always a problem during hands-on workshops!). About 70 people had signed up to attend (a.k.a. about 7 times too many), thankfully other members of the OpenStack community stepped up and offered their help as mentors - thanks again everyone! In the end, about half the participants showed up in the morning, and we lost another dozen to the Internet woes. The people who stayed were mostly enthusiastic and seemed happy with the experience. According to the session etherpad, at least 5 new contributors uploaded a first patch :) Three are merged so far.

Distributing the slides early proved popular and useful. For an interactive workshop with lots of links and references it's really helpful for people to go back on something they missed or want to check again.

Issues

The start of the workshop is a bit lecture-heavy and could be titled "Things I Desperately Wish I Knew When Starting Out," and although there's some quizzes/discussions/demoing I'd love to make it more interactive in the future.

The information requested in order to join the Foundation tends to surprise people, I think because people come at it from the perspective of "I want to submit a patch" rather than "I am preparing to join a Foundation." At the hackerspace sessions in particular (maybe because it was easier to have candid discussions in such a small group), people weren't impressed with being forced to state an affiliation. The lack of obvious answer for volunteers gave the impression that the project cares more about contributions from companies. "Tog" might make an appearance in the company stats in the future :-)

On the sign-up form, the "Statement of Interest" is intimidating and confusing for some people (I certainly remember being uncertain over mine and what was appropriate, back when I was new and joining the Foundation was optional!). I worked around this after the initial session by offering suggestions/tips for both these fields, and spoke a bit more about their purpose.

A few people suggested I simply tell people to sign up for all these accounts in advance so there's more time during the workshop to work on the contribution itself. It's an option, though a number of people still hit non-obvious issues with Gerrit that are difficult to debug (some we opened bugs for, others we added to the etherpad). During one of the pilot sessions at the hackerspace, 6 of the 7 participants had errors when running git review -s  - I'm still not sure why, as it Worked On My Machine (tm) just fine at the same time.


Overall, I'm glad I did this! It was interesting to extract all this information from my brain, wiki pages and docs and attempt to make it as consumable as possible. It's really exciting when people come back later to say they've made a contribution and that the session helped to make the process less scary and more comprehensible. Thanks to all the participants who took the time to offer feedback as well! I hope I can continue to improve on this.

Leave a comment

Python meet-up, November 2012: DevOps 101 DevOps 101

Shame on me for not blogging about PyCon Ireland this year, another success where I happily spent a big chunk of time greeting attendees at the registration desk and attended some interesting talks.

The stylish volunteers:

PyCon Ireland 2012 Volunteers

In the meantime, I took better notes at the November Python Ireland meet-up, where Ulrich took us through DevOps 101, going from the basics of what it is and all sort of related tools to help, mostly Python-based.

I thought interesting that Ulrich worked from a specific definition of DevOps that differs from the ones I heard. There was a time when I used to call myself "a devop" because in addition to development work I was also maintaining production and test servers and handling deployments, etc. However at the time I learnt from other sources that "a devop" was actually a system admin who also writes code to make their job easier... so I just stopped using the word ;) Ulrich based his talk on the premise  of development and operations working together.

Development tends to move fast while operations are more stable and conservative, why would you put these groups together?

The need to scale is an important one. You need the knowledge that the developers have of the application itself, but you also need to understand infrastructure. For instance if you deploy something and suddenly the number of requests spikes, you want devs and ops to be close together to solve the problem.

Following a "release early, release often" principle where you might deploy several times a day makes that collaboration important.

Environments are complicated and it's important for people to integrate ; if the departments remain separate people tend to stick only with the same group of people.

Devops enable better communication and understanding. Developers can move fast and deploy faster as they know what ops need. This also helps mitigate risk thanks to the regular deployments: fewer changes are deployed at once (helpful to determine what breaks) and it becomes easier to do.

The talk then moved on to tools, including advice on how to create a simple django app to keep track of which versions of software are deployed where -- very simple and yet usually enough.

Some of the projects that were mentioned:

  • Using Sword instead of puppet (...which is still best, apparently) if you want to stick to Python, though it's a very young project. Using such tools you can start versioning your configuration and environment, and store reasons for changes in commit messages.
  • Fabric for ssh the Python way :)
  • Virtualenv to isolate Python and manage dependencies
  • Buildout is like Maven for Python projects. It's used to maintain dependencies and deploy, though it can be overkill for many projects...
  • Augeas is not Python specific and supports many configuration formats

Ulrich then went through some of Martin Fowler's principles for refactoring among other things, and also talked about Jenkins, not Python but known as the "Wordpress of Continuous Integration" because it has so many plugins. Sonar can be used for code quality.


We tried something different for the second part of the evening and watched a video of a Pyramid talk. I don't think it quite worked, even at 30 minutes it felt too long. Perhaps we should try again with a video more likely to appeal to a wider range of people, or simply shorter talks. Fair play to Diarmuid for standing up and taking the heat of answering questions afterwards ;)

Leave a comment

EuroPython 2012 Let's go

I'm very happy to be going to EuroPython again this year :) Once more the talks I'm most looking forward to are mainly related to scalability, perhaps with a dash of internationalisation/encoding on the side. The tutorial on Django testing with Selenium should be interesting too.

Say hi if you see me! :)

Python Logo

Leave a comment | 2 so far

Python meetup, June 2012: Message brokers & Python ActiveMQ

Last Wednesday was June's monthly Python Ireland meet-up, this time with a talk by Fernando Ciciliati about message brokers and Python.

After a general  introduction on the advantages and concepts behind messaging, the main part of the talk was about ActiveMQ, a message broker under the Apache umbrella. Fernando explained that this was a broker mostly used by Java people and required a bit of work on their part to get it to play nicely with Python. They had to make significant changes and improvements to the existing libraries.

ActiveMQ is written in Java and designed for high-performance as opposed to reliability, and the default configuration reflects this. If you want your messages to persist after reboot and never be dropped, you will have to tweak the default setup.

The community is very active... which also means lots of bugs! Fernando recommends reading the tracker, as there will certainly be a couple of bugs that are relevant to your situation.

Thanks to the provided binaries, ActiveMQ is easy to install. You get a nice web interface to manage and monitor your queues and messages.

By default, STOMP is not installed and ActiveMQ is only configured to talk to Java via OpenWire. Enabling STOMP support is simply a matter of adding one line to an XML configuration file, and is the easiest way to communicate with Python.

There are several Python libraries available: stomp.py, stompy, Twisted also supports it... They selected stomp.py.

To test your ActiveMQ setup, you can simply telnet into the port and send messages, while using the web interface to see what's happening. The protocol is similar to SMTP.

The terminology used by ActiveMQ differs a bit from other message brokers, be careful when comparing them. (I wonder if this is due to using STOMP vs. using AMQP?). For instance, "topics" seem to be a way to broadcast in ActiveMQ, rather than a way to filter interesting messages.

As usual, the technical talks continued in the pub afterwards :)

Leave a comment

Teaching programming to beginners Latest roundup

An overdue post on the latest course I taught, which ended about 2 weeks ago. I taught a group of 9 people (8 adults, 1 teenager) of various backgrounds the basics of programming, using Python. It went well overall, and the class atmosphere was enjoyable.

For...

Unfortunately I still fail to explain some concepts clearly -- namely the dreaded for loop. I kept telling myself I needed to change the way I introduce it and finally did so, and wow. It went even worse than before. I could see in my student's eyes how they were trying to understand, until eventually shrugging it off when I couldn't find any different way to explain it. It may have come across a bit better eventually when we started working with files and used for loops to go through every line, but introducing the concept went very blergh. Definitely need to do it differently next time. I think I will try to introduce the for loop over 2 sessions: one where we only see and do exercises related to "for i in range(0, x)" and separately introduce "for item in my_list".

I don't know if I just forgot before, but failing to get a concept across is really, really depressing. I was bummed out for a while afterwards, wondering if I'm just making more damage than good and messing up my goal of making programming less intimidating.

...Yet another reason to do better next time!

Exercises

I should make sure to have at least one "Fill in the blanks" and "Change this existing program so that..." type of exercises for every concept. I think they encourage students to read and try to understand existing code -- an important skill to develop, and it also makes the first attempt at using a new structure more manageable.

In general, I'd really like if I could find a neat program that could be cut down into exercises relating to each concept, that'd build up to something cool and awesome as the course goes on. It would help get across that complex programs are really made up of simpler pieces ; if you can't figure out how to do something, break it down further. I haven't really figured out a program that fits the bill yet.

Overall, material-wise I'm getting there (except for For :|), it'd be worthwhile focusing on improving the exercises for the next course.

Homework

OMG some people actually did the exercises as homework this time :O Wow. I tip my hat to them, it's great and it paid dividends for them. Unfortunately it also meant they had nothing to do while the others were catching up, so... I should probably give away only a portion of the exercises as homework, and save a few for the actual session. (The idea with the exercises is that everybody should get at least 1 done before we move on ; having many gives the faster students something to do while others don't feel too rushed.)

Duration

I taught over 5 weeks this time (2h30 in the evening with a 20-30mins break), and I will do 6 weeks next time. For real beginners it's still reasonable and it will let me take more time on concepts like For, and maybe an opportunity to show at the end what else there is to programming, and give ideas on which direction they could go on from here (GUIs, web development, robots...).

Speaking of robots

At the last session I brought a Finch, which I still haven't had much time to play with :') But with the Jython bindings it was possible to show a 5 lines Python program that has an effect in the real world. The Finch was very popular!

I want to play with the robot more, and see if perhaps it would make sense to use it to teach the whole beginner course at some point in the future (which would bring its own set of issues, $$$-wise).

Student feedback

As usual I asked if students could fill in a feedback form.

Everybody agreed they learnt a lot. "Fun" and "interesting" were in about everyone's "3 words to describe the course", with"hard" and "challenging" occasionally thrown in there :) "Useful" came up in half the forms too, which makes me happy.

The projector wasn't great with yellow which unfortunately was the default colour for strings in the IDE. It would have been worthwhile spending the time on fixing that rather than let people squint (sorry :().

A couple of requests to link more clearly exercises with the real world. So very difficult when we're still introducing new concepts! But definitely something to strive for.

Someone suggested writing a sample program where every line is explained (e.g. if x:    <-- this will do...). I think it's a good idea, I will try for next time. It would be useful as a reference.


I should be teaching this course again around March. Any pointers to beginner exercises, general comments, teaching advice, teaching programming tips all very much welcome!

Leave a comment | 5 so far

Teaching, take 4

As mentioned before, I'm teaching programming again, to a group of beginners :) (Although as usual, even in a group of complete beginners there are variations in people's levels and comfort with the subject.)

Initial advertising

I'm really happy with how the PR worked out this time. The first time I taught this course in Tog, I made the mistake of putting "Python" in the name of the course and ended up wasting a lot of time screening out people who wouldn't be a good fit. This time I only mentioned it in passing, and the amount and type of answers I got was much more manageable and appropriate. We still managed to fill up the course and to have a waiting list (hope to see you next time!), so it's a success. I'm pasting the announcement here for future reference:

Class: Introduction to Programming

Are you curious about programming? Ever wished you could write your own scripts to automate repetitive tasks, or for fun? Or been interested in understanding how an application works (and why there are always bugs in the software you use??). Tog is running a new class on learning how to program, for complete beginners with no previous programming knowledge.

The course will cover basic programming skills, which will also give you a better understanding of how computers work. We will be using Python, an excellent learning language because it is simple yet powerful, and extremely readable. At the end of the course you will know how to write simple programs, and you should have enough understanding of the basics to move on to either more complex programming tasks in Python or learn another language if you wish to.

The course will begin on October 20th and will run for 5 weeks every Thursday from 7.00pm to 9.30pm. The cost is €40 for non-members, and free for members. The class size is limited to about 8 people, for a relaxed atmosphere conducive to learning.

If you are interested in attending, or have any question about the course, please use the form below or leave a comment — we’re happy to answer any query! You must sign up if you want to attend, space is limited.

Please note that this is a course for complete beginners, who are not yet familiar with the usual syntax for programming. If you’re a programmer interested in picking up Python, you should keep an eye out for our crash courses!

A fair percentage of the students heard about the course from Twitter even though I'm not on it, which is interesting. I should explore how to use the tool better. As I mostly hang out with fellow nerds, reaching out to interested beginners is always tough, so anything that can help...

Pace

The pace of the course is manageable as well -- which doesn't mean preparing and teaching isn't time-consuming still! But I felt quite burnt out last time after the pace at which I taught the PhD students (mostly through my own fault), so it's refreshing to come back to teaching pure beginners and not go crazy from week 2.

Surprisingly enough and to my utter delight, a huge portion of the students took on the exercises as homework! I'm so happy when I get questions about the exercises or lessons via email, to see people interested enough to poke at problems outside of the classroom. If we keep it up we should be able to cover more as well, which will be interesting for everyone.

As for feedback, I found a couple of positive tweets after the first session, which is always encouraging. Woohoo \o/

Leave a comment

PyCon Ireland 2011 Another successful event!

Congratulations to Python Ireland and the PyCon Ireland committee for the successful 2nd edition of the conference! If you weren't there, you should feel sad. I would be.

Lots of interesting talks and just like last year, a whole bunch of very friendly and knowledgeable attendees to talk to and share a meal with (or a drink, for lucky people who won several raffles... :o)) (hehe) (I did!)

I sadly missed the first keynote, which I look forward to catching up on on video. I was busy helping out at the registration desk, and discovered I really enjoyed welcoming attendees to the conference. A new hobby!

I don't know how productive in general Sunday was, sprint-wise and code-output wise. The open space format seemed to work well on the other hand! I attended the RSI talk and the buildout tutorial (must look more into Buildout) and spent the rest of the day PRing for Tog, distributing Berocca and talking shop. On Sunday afternoon, together with a fellow Tog member we stealthily stole away a dozen attendees to go and visit the hackerspace, conveniently located right behind the venue. Delegates were returned to the conference unharmed and inspired (I hope!).

And because one cannot ever have enough Python, the usual monthly meetup is on this Wednesday!

Leave a comment | 2 so far

PyCon Ireland 2011: BEGIN!

PyCon Ireland 2011 starts tomorrow! If you haven't got your ticket yet, that's tough because we sold out a few days ago! Congratulations, organisers, for what promises to be a fantastic event.

I also seem to have inherited a yellow staff tee-shirt so... Feel free to find me and say hi if you're lost (or not!). I'll hand wave and do my best to be helpful. :)

Leave a comment

Teaching again

Curious about programming? Ever wished you could write your own scripts to automate repetitive tasks, or for fun? Or been interested in understanding how an application works (and why there are always bugs in the software you use??). I'm teaching a new class in Tog on learning how to program, for complete beginners with no previous programming knowledge. We'll be using Python, and you can find more information including how to register over at Tog's website: Class - Introduction to Programming.

Leave a comment

Using Coverage to generate reports (Python)

A couple of tips so I don't have to look it up again. A few months back I wanted to generate coverage data for a stand-alone application, with a few tweaks.

Combining the output of Trial and Python unit tests

$ coverage run -p tests/my_python_tests.py
$ coverage run -p tests/my_trial_tests.py
$ coverage combine
$ coverage html -d tests/coverage_html  --include=./*py --omit='lib/*.py,tests/*.py'

Helpful resources to get used to combine:

Getting coverage report for a simple project

$ coverage run tests/python_tests.py
$ coverage report --include=./*py --omit='lib/*.py,tests/*.py' # For a text summary
$ coverage html -d tests/reports  --include=./*py --omit='lib/*.py,tests/*.py' # For a nice HTML report

Installing the latest version

To get access to the command line tools, I had to install Coverage in a virtualenv to get around some other constraints on the system. I thought I'd build it from master while I was at it :)

$ virtualenv coverage
$ cd coverage
$ source bin/activate
$ pip install hg+http://bitbucket.org/ned/coveragepy

Coverage is a lovely library and a wonderful way to encourage yourself to think more about your unit tests -- give it a try if you haven't already. (PS: For Django, see django-coverage)

Leave a comment

Python/Django Crash Course in Tog

Are you a programmer interested in picking up Python and Django?

Rory is running a crash course in Tog starting this Thursday. There'll be two 2-hours sessions -- the first one about Python, next week about Django.

It's short notice but if you've been meaning to learn Python, now's your chance!

More information, including how to register, over here.

Leave a comment | 2 so far

EuroPython 2011: Nicholas Tollervey on the London Python Code Dojo

Link: Talk description and video

The Python Code Dojo is a community organised monthly meeting.

Dojo

A dojo is a place where you go to practice stuff, learning is a continuous process. It's based on the idea of deliberate practice.

Paris

Codingdojo.org was started in Paris, where it follows a very structured format.

Katas are forms that you practice to prepare yourself. You learn how to solve a problem using baby steps. In Paris they do this in silence, unless you really don't understand and have to ask a question. "Randori kata" is public pair programming, with a pilot and a co-pilot that solve a problem on stage.

London

The London Dojo works more like a seminar and attendees are encouraged to interrupt. Participation is expected. They do team dojo where the team must solve a problem within a timeframe. Problems are written on a blackboard, people vote for one and then everyone works at solving it in a team of 5 or 6 people over 1h30. Finally there is a show, tell, review and question event where each team presents their solution/approach.

Why participate in a dojo?

  • The educational benefit, of learning by doing
  • You can fail safely in a sympathetic environment, and experiment
  • People teach one another, all levels can attend
  • You build a community: in London, that's relaxing with pizza and beer

What's a good dojo?

From the attendee's perspective: it's fun, you get to solve problems, it's safe to make mistakes, show and tell is encouraged which is good to get feedback.

From the organiser's perspective: it self-organises, mostly.

To see if it's going well: see that there is a positive aim, something is done to reach this aim, with some sort of feedback at the end.

Personal observations

Beware of systems and gurus. Ignore systems if something else works for you, you can actually do damage otherwise. Learn to practice learning!

Q&A tidbits

When they (or another dojo?) started using meetup.com they doubled their numbers! Or EventBrite, the idea is to have a centralised system, with tickets to predict attendance.

Leave a comment

EuroPython 2011: Lightning Talks

The lightning talks were very fast paced (5 minutes) so I only jolted down some project names I want to check out and interesting tidbits, and missed speaker names just about all the time. Sorry!

To easily create diagrams, check out blockdiag. It includes different shapes that make me hopeful it might be a less painful way to do nice topologies.

Someone's project to remove the GIL in Pypy (future):

global_lock.acquire() / .release()
object.acquire()
with Transaction():

Learning a language in 60 hours: http://sotospeak.se/ (English homepage). It's a piece of software for your mobile phone, that encourages you to learn a language like children do. It's written in Python.

shlex for simple lexical analysis.

Python Edinburgh are a bunch of cool folks with their own conference :)

DjangoZoom, effortless deployment for Django (like Heroku?)

pip install null, if a need to use the Null object pattern arises.

Leave a comment

EuroPython 2011: Mark Ramm on A Python Takeover

Link: Talk description and video

2 years and a half ago, SourceForge was all PHP except for one little Python service. SourceForge was originally written in 1999, when Python wasn't so great at the web. After 10 years though, it all started to atrophy.

So SourceForge decided to do a little experiment and assigned 2 people to create FossFor.us. They chose a web framework, thinking that Python was good and impressed with the Django documentation: there was no real learning curve. Couchdb was a bit slow, though.

The experiment was a success (although the site is now gone, due to managerial reasons if I understood correctly) and so they were then given 6 weeks to redo the download page/files of SourceForge. It had to be dynamic, to offer you a download relevant to your platform: they use the user agent string to figure out the operating system, and borrowed code from setuptools to figure out which release is the latest.

Their admins loved Apache so they went with mod_wsgi, which worked out well.

The whole system worked fine on a laptop handling all the traffic from SourceForge: that was the only load testing! And unfortunately when the system went live it took about 8 seconds to load a page. There was no CPU or memory usage. It turned out they saturated their gigabit Ethernet card, by loading all the releases, which some projects have a lot of (e.g. JBoss). Memcached turned out to be slower than MongoDB, because of pickling, the 4 megabytes objects and CPU.

Finally they updated the list to not include all releases and the project was deemed a success! Thus everything shall become Python. They are developing their new platform openly, Allura. They use a FUSE filesystem in Python to control permissions.

From now on, they have an internal mandate that everything should be written using Python unless there is a good reason not to. And now SourceForge can compete again! As well as explore new directions.

Leave a comment