I am the new CPython Developer in Residence
This is some of the most amazing news in the past few years for me. Python needs full-time development to stay competitive, I’ve been talking about this for years, dreaming about it for even longer than that. Now it’s becoming a reality. Today is my first day. It’s both scary and exciting.
When the PSF first announced the Developer in Residence position, I was immediately incredibly hopeful for Python. I think it’s a role with transformational potential for the project. In short, I believe the mission of the Developer in Residence (DIR) is to accelerate the developer experience of everybody else. This includes not only the core development team, but most importantly the drive-by contributors submitting pull requests and creating issues on the tracker.
Before we delve into details on what that means for me, I’d like to express my gratitude to the Python Software Foundation and the Steering Council for their trust in me. Applying and interviewing for the position were great experiences, I’m sure we’ll do some great work together.
Tapping into External Contributors
Let me elaborate what I meant by “developer experience acceleration”. Python is developed by volunteers and there’s few people like Victor Stinner paid to do full-time maintenance through their employers. Now that we have a team led by Guido at Microsoft to speed up Python, just the three people involved are already visibly more active than a large part of the team who are volunteers in their free time. Just by the sheer force of sitting at a desk for a given number of hours they can achieve big things.
Now, what can the DIR do as one person? I believe I can multiply the impact of the hundreds of contributors who are not core developers. The DIR can do it by:
- providing a steady review stream which helps dealing with PR backlog;
- triaging issues on the tracker dealing with issue backlog;
- being present in official communication channels to unblock people with questions;
- keeping CI and the test suite in usable state which further helps contributors focus on their changes at hand;
- keeping tabs on where the most work is needed and what parts of the project are most important.
There is an important side effect to providing this service, and that is a good first developer experience for an external contributor. Great contributor experiences lead to future contributions, and a stream of contributions leads to an occasional contributor becoming a core developer. I’ve seen this through leading the Black project. It’s now maintained by 9 people including me.
Helping Out Momentum
The Steering Council has ideas where Python should go. Existing core developers already have plans for PEPs and large-scale changes. There already is momentum in terms of where the project is headed. And while I have some ideas of my own, I believe the DIR is not supposed to be throwing his ideas around like some self-proclaimed “CEO of Python”. Instead of telling others what to do, the DIR is meant to be a steward, sometimes a janitor, to help accelerate existing momentum, unblocking progress and ensuring changes can be implemented in time and with sufficient quality.
For instance, providing an additional pair of hands to help discovering and addressing regressions in rapidly developed changes, like the current push for performance, is going to have tremendous impact on the speed of Python 3.11 but also its stability and backwards compatibility.
The PSF and the Steering Council summarize the expected responsibilities of the DIR as follows:
- addressing PR and issue backlog,
- analytical research to understand the project’s volunteer hours and funding,
- investigation of project priorities and their tasks going forward,
- and working on those priorities.
Addressing the backlog will require creating a long-term plan for how to manage that going forward. In practical terms, there will be a lot of personal pull request review and issue triaging, as well as coordination with other core developers/maintainers of specific modules to solve issues and merge pull requests.
One important piece of the puzzle is improving, stabilizing, and maintaining the test suite and the CI that runs it, including buildbots. Making sure changes going in are good by having fast and reliable CI is one of the most direct ways in which the DIR can positively affect the developer experience for the rest of the team.
Research to understand the project better can be done by capturing user interest in particular libraries within the standard library, the amount of work given libraries require, and who the active experts behind the libraries are. This can be done through open issue analysis, the amount of pull requests for a given library, but also through surveying core developers and users as some libraries might require little maintenance but be critical. The reason to do this research is to determine which standard library modules need help and what the maintainer cost is for standard library modules.
Measures of Success
Since this is going to be a pretty open position, both in terms of work style as well as in the sense of visibility to everybody, I guess the most important thing to determine is how to measure the role’s performance.
It’s clear that consistency, transparency, and visibility will be key. That includes using the official channels for communication in the Python project, as well as attend Steering Council meetings and have regular communications with the PSF staff. Additional progress reports in the form of blog posts on pyfound.blogspot.com were suggested by the PSF.
Some basic metrics that I think should be measured and reported as work goes on include the number of open PRs and issues, number of commits compared to same time during previous release, number of contributors, number of PEPs merged over a period of time, and so on.
But there’s more interesting metrics below the surface that are less obvious in terms of measurement as well as quantifying the DIR’s influence on them. It would be awesome to have some insight into whether having a DIR improves the health of the Python community, has any concrete impact over Python adoption, or Python runtime’s performance, and so on, and so on. How to measure this is admittedly unclear to me at this point.
While the position itself is a new thing for CPython, it’s not a new idea in the open source world. The Django Fellow is a similar concept, successfully ran since 2014. In fact, I’ve been talking to Mariusz Felisiak, the current Django Fellow, to get a sense of what this kind of role entails and how it is performed day to day.
Many of the things I write about above have been confirmed in those conversations as sensible. In particular, the Django fellowship explicitly says that it’s about “the work that benefits most from constant, guaranteed attention rather than volunteer-only efforts”.
This is exemplified by Django Fellow weekly activity reports which I find excellent as they show steady progress. I wonder if the weekly averages of 15 triaged issues, and 12 reviewed+merged PRs, and 3 authored changes are going to translate to CPython but they definitely provide ballpark values. Similarly, Anthony Shaw working for Microsoft’s Azure Open Source team hit 100 merged PRs in 6 months which averages over 4 PRs every work week.
When I applied for the DIR position, I did it after quite a bit of soul searching. On the one hand, I think I’m a good fit for it through my long-term involvement with Python core development on multiple facets:
- code and design contributions (over 5 approved PEPs);
- management contributions (release management, Summer of Code student mentorship, organizing core sprints and the Language Summit);
- teaching and developer advocacy (public talks, facilitating discussions on Discourse).
On the other hand, this year will prove whether this position is worth keeping for future years. I deeply believe it is, therefore it all comes down to my performance over the next twelve months. I treat this seriously but also personally. I mean, I am very deeply invested in the Python ecosystem and community. My biggest career accomplishments are related to Python and my strongest friendships and professional relationships were formed around the Python community. As a release manager I was trusted by senior members of the project with a task that involves commitment for over 8 years. I will be around for years to come. The DIR position is an entirely new mode of contribution though. It requires consistency and requires much more breadth. I feel this responsibility. I’ll do my best not to disappoint.