Simply Jonathan

Archive for March 2018

Is Python Interpreted or Compiled? Yes 

Ned Batchelder on the grey zone between ‘interpreted’ and ‘compiled’ languages where Python (and most programming languages, really) is.

User-defined Order in SQL 

This is the sort of problem which on the surface seems so simple, but where the immediate solution is so wasteful, and can indeed be hard to pull off; my side project Oversigter has been doing this incorrectly for years.

The solution is elegant, although it can be difficult to pull off in practice if you don’t have a lot of control over your hosting environment, as I don’t where Oversigter is hosted.

Via Simon Willison

Firefox Facebook Container Extension 

While I applaud treating Facebook as a virus to be contained, I somehow doubt a similar version for Google is right around the corner.

(Via John Gruber)

Checking for Missing Migrations in Django

Back in the day, Django didn’t have a built-in way to change model schemas. You either had to figure out and apply the changes yourself or use a third party tool like South.

After a successful Kickstarter campaign, Andrew Godwin, Django core member and the author of South, added a native migration tool to Django. It solved problems such as conflicting merge names and the ability to squash migrations once there are too many of them in an app.

One thing it also changed is introduce what I have dubbed ‘cosmetic migrations’: these are migrations that make no changes to the database schema, but only add internal Django changes, such as changing a field’s choices or the ordering of a model. I’m sure the change is for good reason, but it annoys me to no end, because the lack of schema impact means I’m unlikely to notice that I haven’t made them until at some point I do make a schema-altering change, and that migration is then flooded with an untold number of cosmetic changes. This is a problem because commits should be atomic.

Django will occasionally notify you that ‘[y]our models have changes that are not yet reflected in a migration’, but I found that I would only see those when it was too late.

The solution

Thankfully, it is possible to make these checks yourself, although I have never seen it advertised anywhere. Executing this command will give your what you need:

$ python manage.py makemigrations --check

This command seems wholly counter-intuitive to me, but it does what I want: Exit with a code 1 if there are unreflected changes.

You can plug the above in to your Continuous Integration system or possibly a pre-commit hook. If you do so, I recommend that you also make it --dry-run, like so:

$ python manage.py makemigrations --check --dry-run

This will ensure that no migrations are actually created, which just seems the saner option if you want to check.

Facebook, Always There to Help

A screenshot from Facebook with the message, 'Jonathan, it's almost your birthday!'

Thanks, Facebook. Thanks for also reminding me what my name is, you’re a pal.

(Yes, they did use it as a call to action to donate to a charity, but I still find it a ridiculous hook.)

What’s the Cheapest Empty Iterable in Python?

From the department of ‘optimisations that are so premature that if you ever find yourself actually caring about it, you need to find better problems to solve’, I recently had this thought: What’s the cheapest/fastest iterable in Python?

Reminder: iterable and iterator are not the same thing.

The need for an empty iterable occasionally comes up, like when you need to provide a default value to a missing key in a dictionary, and you need it to be something you can iterate over without running the risk of a TypeError. The beautiful thing, of course, is that you can iterate over an empty iterable and just have nothing happen, so the actual type or contents don’t matter.

So I set out to test it. Again: you should never need to actually care about this. If you can live with the actual overhead of iterating over something, you can live with the overhead if that something is empty, no matter the actual type of iterable.

I evaluated strings, lists, tuples, dictionaries and sets. My hypothesis was that the fastest would be a string or maybe a tuple.

The test was performed on a late-2016 13″ MacBook Pro with a 3.3 GHz Intel Core i7 and I timed it using the timeit module.

First I tested out simply declaring the different types of iterables:

kweli:~ j$ python -m timeit -c '""'
100000000 loops, best of 3: 0.00672 usec per loop
kweli:~ j$ python -m timeit -c '[]'
100000000 loops, best of 3: 0.0187 usec per loop
kweli:~ j$ python -m timeit -c '()'
100000000 loops, best of 3: 0.0119 usec per loop
kweli:~ j$ python -m timeit -c '{}'
10000000 loops, best of 3: 0.0305 usec per loop
kweli:~ j$ python -m timeit -c 'set()'
10000000 loops, best of 3: 0.0924 usec per loop

So far so good: strings are the fastest, followed by tuples and dicts, with sets trailing far behind.

Then to actually iterating over them:

kweli:~ j$ python -m timeit -c 'for i in "": pass'
10000000 loops, best of 3: 0.0433 usec per loop
kweli:~ j$ python -m timeit -c 'for i in []: pass'
10000000 loops, best of 3: 0.0514 usec per loop
kweli:~ j$ python -m timeit -c 'for i in (): pass'
10000000 loops, best of 3: 0.0438 usec per loop
kweli:~ j$ python -m timeit -c 'for i in {}: pass'
10000000 loops, best of 3: 0.0707 usec per loop
kweli:~ j$ python -m timeit -c 'for i in set(): pass'
10000000 loops, best of 3: 0.136 usec per loop

And again, the hypothesis is confirmed, but interestingly the difference between lists and strings/tuples is much smaller when iterating compared to just declaring.

So in conclusion, use a string as an empty iterable, unless you have any reason at all not to. The difference is infinitesimal.

Two Cultures

I was in New York in February, where it was unseasonably warm. So warm that you could see stuff like this in Prospect Park:

A sign that reads 'Danger: Thin Ice' in front of a completely non-frozen lake
Thin ice indeed

Yesterday I was out walking around the lake Damhussøen in my native Copenhagen. Here it is unseasonably cold for March (although not in the what-the-fuck-it’s-68°f-in-New-York-in-February category), and the lake is completely frozen:

A completely frozen lake, complete with a tree branch comfortably nestled on it; no signposts

But in Denmark, you’re on your own. To be fair, based on tracks we saw, it seems that people could actually walk on Damhussøen, though I doubt it’s something anyone would recommend.

This is Simply Jonathan, a blog written by Jonathan Holst. It's mostly about technical topics (and mainly the Web at that), but an occasional post on clothing, sports, and general personal life topics can be found.

Jonathan Holst is a programmer, language enthusiast, sports fan, and appreciator of good design, living in Copenhagen, Denmark, Europe. He is also someone pretentious enough to call himself the 'author' of a blog. And talk about himself in the third person.