As I added an is_valid column to my Imageset table, suddenly all queries with a NATURAL JOIN imageset started to break unit tests.
I learned that it's because Natural joins simply join tables based on column names and the Image table happen to already have an is_valid column.
NATURAL JOIN does not look for a match with the primary key first, then foreign keys. It just takes the first common name and type between the 2 tables. In my case it was joining on is_valid while imageset_id is the primary key of the Imageset table and also the foreign key between the 2 tables.
Mind. Blown.
The choice is probably deterministic so it's not really any key but I'm too lazy to look at the MySQL implementation.
The takeway here is to use INNER JOIN instead.