Django currently supports Python 3.6+, on Linux, Windows and macOS. So checking e.g. the Summer of Code project to adjust the parallel test runner to support OSes which use the fork method for subprocesses (specifically Windows and macOS from Python 3.8) requires running a whole load of different combinations.
On Windows I use the official installers from python.org, I don’t put python on my PATH, and then use the py launcher to switch between them. This works very well, and I highly recommend it.
You can get Python from the Windows Store now, which is amazing, but I’ve not yet got to the point where I’m 100% confident about what goes where, and how the store builds interact with the py launcher to say Use that! — it’s certainly a good way to get started if you just want one version of Python though.
On macOS (and Linux) I use pyenv to manage Python versions and switch between them. It’s great. Very few problems.
Except...
We also need to test various SQLite versions. Python 3.6 ships with SQLite version Not.Very.High, whereas Python 3.8 gives us version Totally.Up-to.Date. So we have to make sure we’re compatible with a range there too.
For macOS you can (or could) specify a path to a given SQLite version in order to use that. You'd compile your version and then, set an environment variable:
DYLD_LIBRARY_PATH=path/to/sqlite ./manage.py tests
On Linux you’d do the same but with LD_PRELOAD.
The Window story involves replacing the SQLite library inside your Python install directly: if someone knows a different way, I’d love to hear it!
Except, much messing about this week later, macOS’s System Integrity Protection broke this usage, by stripping all DYLD prefixed variables from the environment of any process in a whole range of protected areas.
So where we’re expecting to be able to set DYLD_LIBRARY_PATH we can’t, and there’s much head-scratching as our target SQLite version isn’t used:
~ $ DYLD_LIBRARY_PATH="testing" python -c "import os; print(os.getenv('DYLD_LIBRARY_PATH'))"
None
Pyenv manages my Python installs. These all live in my home folder, and in theory aren’t protected by S.I.P.
But (here’s the rub) in the command above python is the Pyenv shim, and this uses /usr/bin/env which is so protected:
~ $ which python
/Users/carlton/.pyenv/shims/python
~ $ head -1 $(which python)
#!/usr/bin/env bash
In order to get around S.I.P, then, I need to invoke the python executable directly, rather than go via the shim:
~ $ DYLD_LIBRARY_PATH="testing" $(pyenv which python) -c "import os; print(os.getenv('DYLD_LIBRARY_PATH'))"
testing
OK. We’re in action. (As I say, much head scratching.)
It’s not clear to me whether pyenv can work around needing to go via /usr/bin/env, but needing to update the SQLite version is a relatively common use-case (I’d say.)
It’s worth noting that recent Python builds from python.org enable a flag to not strip DYLD environment vars, but again this won’t work if you went via /usr/bin/env which will have already done it.
The new Django Technical Board was elected this week. There was some discussion that (still/again) all the candidates were male. At the moment there’s not much we can do about this: the TB by its nature requires involvement in Django’s development, and it’s still the case that regular contributors on GitHub and Django-developers are almost all men. (The “almost” there is overly generous…)
This is something that’s been on my mind for a while, at least since the beginnings of the Dissolving Django Core conversations, which eventually led to the change of governance in DEP 10. It’s not something I’ve had too much time to think about in the last year but it came back to mind recently reading the (highly recommended) Working in Public book, and then again with the TB election. It’s something I want to push forwards with over the next period. Have a hobby, they said. Mine’s GitHub and Django, and this is the bit where it’s lacking.
Two things I’d like to focus on, recognising more contributors, and widening the definition of what counts a contributor — this last being it’s not just commits, Django being much more than just the code in the django/django repo.
I previously mentioned Button, a simpler deployment story. Coming 2021, Button is my take deployment. It’s part guide, part tool.
Deployment isn’t magic — there’s just a lot to it. What I want to do is document the patterns, so that I’ve got somewhere canonical to point people to, and then add tooling around the more involved parts, so as to smooth the pain. Will Vincent is always telling me off for saying things are simple but, the basics of deployment are pretty simple, and I want to make that plain. (You can follow @btnapp on Twitter if you’d like updates on this towards the end of the year.)
This week I dusted off the Apple Developer hat on the tooling side: updated to the Big Sur beta, got up and running with the latest Xcode and SDK, and knocked around seeing what's new.
I'm particularly impressed with the new iOS 14 widgets: I’m very keen to have my deployment status on my phone’s homescreen. That you can put that exact same thing onto the Mac is kind-of cool, and prompted me to give it a go.
I skipped Catalina entirely. As such I’m enjoying Big Sur a lot. Xcode has come on, and SwiftUI is very interesting from a development point of view. As ever, the usual caveats about installing betas — make sure you know what you're up to, and what you’d do if it all went wrong — but it was Developer Beta 9, by which time you’re kind of hoping the major bumps have been ironed out, and I wanted to start prototyping.
Tom released DRF 3.12. This takes a weight off of me, that I’ve been feeling quite heavily. With a bit more mental space (especially after DjangoCon Europe) I want to turn back now to Channels, which has been dragging on for lack of attention.
Goals is too strong. It implies you might actually get things done. But the key activities for Q4 are these:
All of those on the Ticking Over level. It’s still 2020 remember!