James Crisp

Software dev, tech, mind hacks and the occasional personal bit

AirTag 2nd Generation incompatibility

To see the location of an AirTag second generation, you need to have an Apple device running the latest OS. For some reason, Find My online (website) does not show AirTags. Furthermore, older Apple devices won’t even help you. Sequoia on Mac is unable to see the AirTag second generation location – despite it still being a supported OS version. You need a device running Tahoe. AirTags second generation are surprisingly and unhelpfully incompatible.

The window.beforeUnload event – a sad tale

Well now, gather around children…

Once upon a time, the good developers wanted to save their users from losing changes if they left a form before submitting it. And as sometimes happens, Browsers looked kindly on the world, and provided the beforeUnload event, which could be hooked in Javascript so that changes could be automatically saved or a confirm dialog could be shown. The developers and users were overjoyed, and a golden age ensued.

But then enter the evil developers! These developers hooked the beforeUnload event to make the user’s life miserable. They coded event handlers so that people could never leave pages, and were stuck in a hell of eternal dialogs and never-ending pop-up windows.

The Browsers realised that beforeUnload had been a mistake, and something had to be done. The event was still needed by the good developers, but the evil developers had to be contained. So the event was changed to something “special”, and uncontrollable!

Thus code like this was born:

window.addEventListener('beforeunload', function(e) {
  e.preventDefault();
  e.returnValue = ''; // Triggers the browser's generic warning confirm, cannot be customised
  } 
});

And it mainly worked. The evil developers were stopped, and the users were freed from the inescapable prisons of popups and alerts.

But the good developers wanted to write automated browser tests to ensure that the beforeUnload event worked, and that users would be saved from losing their changes forever. And this is where the tale becomes even sadder.

In Selenium / Firefox, no confirm dialog is shown from beforeUnload in tests, even though the confirm dialog shows during a usual interactive session.

In Cuprite / Chrome the story is more complicated. A message is shown on the console:

Modal window with text `` has been opened, but you didn't wrap your code into (`accept_prompt` | `dismiss_prompt` | `accept_confirm` | `dismiss_confirm` | `accept_alert`), accepting by default

But none of the ‘accept’ or ‘dismiss’ options work, or catch the dialog. They only work on normal JavaScript confirms and alerts. The only option is:

page.driver.browser.on(:dialog) do |dialog|
  # I DO GET CALLED! BUT ASYNCHRONOUSLY.
end

But the dialog object cannot be accepted or dismissed, unlike all other dialog objects. It also has an empty message. Alas, our story ends a testing tragedy. The console warning message is always shown, and no significant test can be written, as the Cancel (dismiss) button cannot be pressed.

And so, to this day, the good developers must test this code manually, like it was the 1990s.

PS – do let me know if this tale is wrong and there is a better way…

PPS – Chrome, Edge and Firefox support beforeUnload in this way. But Safari on iOS ignores it. Safari on Mac largely supports beforeUnload, but after a cancel in the dialog to stay on the page, it doesn’t seem like the page can do a programmatic redirect from JS later (which other browsers allow).

Todays take on the BEST AI for a task…

Coding: Claude 3.7 for front-end work, especially with the help of Claude Projects to upload supporting files, and Extended Thinking turned on. Remember to always ask Claude to explain its plan before doing anything as it can get carried away. Claude 3.7 paid responds much faster than the free version.

Mobile App – Voice: I do like to sometimes ask questions using my voice and have a conversation with an AI who speaks aloud. The ChatGPT app does a great job of this, and has good support for image/video uploads as well (eg, identify this spider please). The other AIs interact in a far less “human” way – more like just text-to-speech. Again, this requires the paid subscription.

Deep research: Yes, it is Gemini 2.5 Pro Deep Research. Requests are limited and it is slow to give you an answer, but quite comprehensive. It writes you a long report but also provides a good summary of findings.

Tomorrow is a new day, expect change!

Web Fonts built into all browsers (for sans-serif)

The fonts built into modern browsers are varied – it is hard to avoid loading your own fonts. Arial is the rare exception – it is well supported across all old and modern browsers. It looks a little different between Windows and Mac / iOS especially as a larger bold heading. On Android, it is substituted by Roboto. On Linux variants, Arial will likely fall back to the sans-serif system font.

Verdana is also well supported but looks a bit odd in this day and age.

You could instead go with the system font on each platform (eg, San Francisco for Mac/iOS, Segoe UI for Windows, Roboto for Android, etc). However, this means that every platform will look a little different, especially for larger and heavier font styles. Personally I like San Francisco and Roboto (and they look fairly similar). However, I find Segoe UI in large and heavy looks quite different and unsuitable for a “serious” website. If you could use Roboto on Windows reliability, this approach would be better.. but Roboto is only included with Edge but not Chrome on Windows.

So basically, if you want a “serious” looking sans-serif font that is similar across platforms, then it’s either Arial OR it’s time to load your own font and take the performance penalty in return for the look and consistency across platforms.

Google Fonts is a good place to get a font to self host. You can then convert the downloaded ttf with woff2compress (also available on brew) to make it a smaller download and host as a woff2. The woff2 format is now supported by all modern browsers.

“The Obstacle is the Way” by Ryan Holiday

After reading an interview with DHH where he mentioned this book, and to assuage a latent interest in stoicism, I decided to check out “The Obstacle is the Way”. I found it an invigorating read. It is in the usual self-help style with stories supporting the ideas and approaches proposed.

Here are my notes on the book:

Be tough, strong and resilient rather than entitled or precious.
Do not give in to adversity, this is true strength.

Observe without judgement or reaction. Maintain sang froid. Be intentional. There is always an action you can take to make it better or worse.

Break it down. Go step by step. Just do the next thing right now. Do it well and with enthusiasm.

An obstacle or set back is a chance to practice a skill, to learn or to get stronger. The obstacle is the path.
Every crisis is an opportunity to do something you couldn’t before.

Accept and acquiesce to what cannot be changed. It is so. Once you’ve done your best, it is up to fate/God. But remember you can still achieve your goals like water always finds its way downhill.

Love your fate. From hard times, there is always a benefit – learn, rebuild better, build character, etc. Maintain an unfailing cheerfulness.

Think: If it happened, it was meant to happen, and I’m glad that it happened when it did. I am meant to make the best of it.

Perseverance in key to grind down obstacles. Churchill’s acronym KBO: keep buggering on!

Even if your can’t help yourself, think how you can help others in a difficult situation. 

After each obstacle is another obstacle. The more you accomplish, the more things will stand in your way. Each one in a new opportunity to develop strength, wisdom and perspective. Be eager for the next round.

“Predicably Irrational” by Dan Ariely

A friend several jobs ago recommended I read “Predicably Irrational” by Dan Ariely. I bought it with all the best intentions, but somehow it sat on my bookshelf for a long time. Having finally read it, I was impressed. It has some well researched and backed up lessons on human behaviour, plus an engaging style.

Here are some of the ideas that stood out for me.

Relativity (always comparing)

A lot of value judgements and decisions come from comparisons, and are hard to make without anything to compare against. This can be gamed by providing decoy options. Eg, a similar item which is a bit cheaper or more expensive. Or when dating, this could be a less attractive friend along with you.

Executive salaries increased by 3 times when they became public, as other execs demanded more. If all company salaries became public, all employees except the top few would be dissatisfied.

Choose carefully who you associate with and what you look at as you’ll be automatically comparing with them.

Also, if buying a pen for $25, and could drive 15 min to another store to save $7, most people would. But when buying a suit for $455, most would not drive 15 min to get the same suit for $448. But it’s the same $7 saving. The proportion really doesn’t matter.

Anchors

For something new, the price of the first item that you seriously consider buying will become the anchor price for all similar items. If you want to sell something new for a high price, show it surrounded by expensive and luxurious things to make it in the same luxurious class of things and command a high price. If you want to sell something that is not that new, differentiate it enough so it seems like a new class of item (eg, Starbucks) so it can be anchored with a new price.

Also, if you have made a decision once, it is easy to continue repeating it without thinking about it again.

Free

Free is magical. People will fill in lengthy forms or wait in lines for hours to get something free. Including something for free in an offer is a very powerful draw, even if it is of minimal value overall.

Social norms vs Market norms

We live in two words, one social exchanges and one market based. Gifts stay in the world of social exchanges. Any mention of money switches to market norms. Eg, you could ask your neighbour to help you move a couch and give a gift afterwards as a thank you. But giving cash would wreck the exchange. Similarly, mentioning the cost of things on a date would be disastrous. Once a relationship switches to market norms, it is very hard to go back. Companies can use social norms to their advantage to motivate employees or for customer loyalty, but need to be kind in return for it to work.

Ownership

Avoiding loss is a much stronger motivator than gain. People overvalue what they own, and don’t want to lose the benefit an item provides. It is very hard to go back once you have “upgraded”.

Beliefs

If you believe something will be good beforehand, your mind will work hard to make you enjoy it to meet your expectations (eg, fine presentation of food). Or vice versa.

Cheating / theft

Most people who consider themselves honest will still cheat / steal a little before they stop themselves (eg, taking a pen from a conference, inflating the value of an item for insurance, eating something from a shared fridge that is not theirs). Thinking about honesty beforehand (10 commandments, an oath etc) makes people more honest. It’s easy to rationalise petty dishonesty. Dealing with cash makes most people more honest (eg, taking a pencil from work is OK, but taking money from petty cash is not).

Solving mysterious null values in Mysql date columns, stored by a Rails app

A few months ago, when I was doing some detailed database backup and restore testing, I discovered there were, out of millions of records which had user-entered dates, a handful that had null dates in a database column. I scratched my head for a while and couldn’t work out how this had happened, as the field is validated in Rails for empty/null.

Just today, I got an exception report from a different part of the system which does a query based on the user entered date, and it revealed the source of this extremely rare problem! So.. drum roll..

Accidentally, a user had entered a date with the year 20223.

This is valid in Ruby/Rails but too big to be stored in the mysql Date column, so had ended up (silently) being stored as null!

Easily fixed by limiting the date range a bit!

Testing performance before upgrading from Mysql 5.7 to Mysql 8

Mysql 5.7 is reaching end of life in October, so it is becoming important to upgrade. I am using Percona Mysql and the upgrade process with Apt and the Percona repositories is simple. Mysql automatically upgrades files and tables to suit the new version too. There are config file changes required, and quite a lot of defaults changed in Mysql 8, but that is not the focus of this post.

Reading up on the performance differences from Mysql 5.7 to Mysql 8, the story is mixed. In my case, Mysql 8 is definitely slower out of the box for queries that are not well indexed. We are talking often about 50% slower on larger selects. With well indexed queries, the difference is negligible. I kept looking for some config setting that would bring back old Mysql 5.7 performance, but I found no way to make Mysql 8 perform as well out of the box. The solution I found was to add more indexes. These indexes had not been required in 5.7 for good performance. In Mysql 8, they proved vital.

Considering the difference in performance between versions, and additional indexes I had put in place, I wanted to test the performance before upgrading my production setup to Mysql 8. A good way to test performance would have been to mirror incoming requests between real production, and a cloned production server, and watch the comparative performance. This is arguable the best option, but requires quite a bit of infrastructure work and HTTPS offloaded from the production server to something that is mirroring the requests and ignoring responses from the clone. AWS supports traffic mirroring, but I decided it wasn’t the best option in my situation as it would have required significant infrastructure and production changes for my setup.

The alternative that worked in my case, was to record all database queries for a period of medium-high site load, and then replay these on clone servers, to test the relative performance between database server versions and index additions. Tools exist to do this, but they are a bit dated.

If you’re interested in using approach, first record all queries to file at a time of significant load, using the Slow Log on your production server (eg, for 10 minutes). Simultaneously, take a snapshot of the server that you can use for creating clone servers at this point in time.

To test relative performance on clones, I used Percona Playback. This is a very handy tool but old and unmaintained – still, it was the best option I found. To make Playback work, you need to run it under CentOS 7, which is easily achieved using Docker. I tried updating the code to run on modern Ubuntu but it was too big a job, libraries it depended on had changed too much.

To install, set up a data directory and grab and build my Dockerfile (I updated it a little from @lichnost’s version):

mkdir -p playback-docker/data
cd playback-docker
curl https://github.com/jcrisp/percona-query-playback/blob/master/Dockerfile > Dockerfile
docker build -t percona-playback .

Transfer your query log (eg, MyServerName-slow.log) to the clone. I’d also recommend taking a snapshot at this point, and creating new clones for performance testing from this new snapshot, since it now has Percona Playback installed and the query log file available.

To replay the queries in full on a cloned database server:

docker run --mount type=bind,source=$PWD/data,target=/app/data  --mount type=bind,source=/var/run/mysqld/mysqld.sock,target=/var/lib/mysql/mysql.sock percona-playback --mysql-max-retries 1 --mysql-host localhost --mysql-port 3306 --mysql-username <USERNAME> --mysql-password <PASSWORD> --mysql-schema <DATABASE_NAME> --query-log-file /app/data/MyServerName-slow.log

Note that this is “destructive”. All updates/inserts/deletes will be applied, so the run is only fully valid once. Re-running it will lead to errors like duplicate ID inserts or deletes of records that don’t exist any more.

For a non-destructive performance test, you can filter out just the selects using another tool, pt-query-digest, which is happily still maintained and can be installed as part of the percona-toolkit package. To make a select only log file:

cat MyServerName-slow.log |  pt-query-digest --filter '$event->{arg} =~ m/^select/i' --output slowlog > selects.log

For performance comparisons, I ran the non-destructive select-only version first to warm up the database before running the full destructive version as the real test.

This approach gave me confidence that, with additional indexes, Mysql 8 would be faster than Mysql 5.7 on a real production load for my app.

Percona MySQL: Collecting All Queries with the Slow Query Log

If you want to collect all queries running on your database for a period of time, for analysis or replay, the simplest way to do this is using the Slow Query Log.

To start collecting:

set global slow_query_log_file='/var/log/mysql/slow.log';
SET GLOBAL slow_query_log=1;
SET GLOBAL long_query_time=0;
SET GLOBAL log_slow_admin_statements=1;
SET GLOBAL log_queries_not_using_indexes=1;
SET GLOBAL slow_query_log_always_write_time=0;
set global slow_query_log_use_global_control="all";

To stop collecting and revert setting changes:

SET GLOBAL slow_query_log=0;
SET GLOBAL long_query_time=10;
SET GLOBAL log_slow_admin_statements=0;
SET GLOBAL log_queries_not_using_indexes=0;
SET GLOBAL slow_query_log_always_write_time=10;

The statements will be logged to:

/var/log/mysql/slow.log

The performance degradation from logging all queries was not significant in my case, with my site under medium-high load.

You may have to vary the slow_query_log_file location in the first line based on your configuration of secure_log_path.

ChatGPT Programming Test

I was writing a little function in Ruby and thought I’d ask ChatGPT to have a go at it.

It is easy to read, explicit, and fairly idiomatic Ruby (though not concise). Not the most performant implementation but nothing terrible. It also mainly works but does have a bug in some cases.

Here is the code:

def working_days_between(start_date, end_date)
  # Make sure the start date is before the end date
  start_date, end_date = end_date, start_date if start_date > end_date
  
  # Calculate the number of days between the two dates
  days = (end_date - start_date).to_i
  
  # Calculate the number of weekends between the two dates
  weekends = ((start_date..end_date).count { |date| date.saturday? || date.sunday? })
  
  # Subtract the weekends from the total number of days to get the number of working days
  working_days = days - weekends
  
  return working_days
end

If you have the start/end date on a weekend, then you get a negative answer. Eg,

working_days_between(Date.parse("Sat, 04 Mar 2023"), Date.parse("Sun, 05 March 2023"))
 => -1

It is because the weekend number of days calculation is including both the start date and the end date. Ie, working_days = 1 – 2 = -1

A human could easily have made the same mistake, mind you.

A better / simpler implementation is:

(from_date...to_date).count { |date| date.on_weekday? }

Note the 3 dots (…) for the date range, which does not include the end date.

Later, I tried asking ChatGPT to regenerate the answer multiple times. It gave me quite a different version every time – some versions with bugs, some with no functions, some with support for public holidays, etc.

Page 1 of 21

Powered by WordPress & Theme by Anders Norén