kellegous.com

Filtering MCP Tools with jq

2026-03-21T00:00:00Z

Last week we were attaching an MCP server to everything; this week we’re throwing them all out and replacing them with CLI’s (Command Line Interfaces). I think this may have all started when Open Claw creator, Peter Steinberger, was on the Lex Friedman podcast and took a subtle dig at MCP by saying “Screw MCP’s. Every MCP would be better as a CLI”. He’s not wrong and he goes on to explain one of the issues with MCP. If you ask an agent using an MCP for weather information about the current temperature, it might call a tool called current_conditions. That tool likely returns a lot of information: apparent temperature, relative humidity, wind speed, visibility… I’m only interested in temperature, but all of that other information ends up in my context window, just burning up precious tokens. You might also be inclined to fix this by making a lot of tools that return small amounts of data. Sadly that results in a lot of tools with similar names and the problem of selecting the right tool becomes the point of frustration. By the way, smarter people have written more elegantly about these tradeoffs. Back in November, Anthropic talked about these tradeoffs in Introducing advanced tool use on the Claude Developer Platform.

So why do people think CLI’s would be better? Well, if you use Claude Code much, you are very well aware that it has an absolute mastery of the unix command line. Rarely does it issue a shell command that is not piped into grep, or jq, or something else that filters or transforms the data from one command into exactly what is needed to to answer the question. When Steinberger says “MCP isn’t composable,” this is what he’s getting at. There isn’t an ecosystem that allows you to send the output of one MCP tool to another tool to be transformed before it ends up in the context window. Even if there was such an ecosystem, it would not be well represented in the training data. Unix shell kung-fu, however, is a skill that seems to be baked into the weights of these models. You don’t even have to waste space in the system prompt, the model just knows this stuff.

Wait. The future is … command line UNIX!?

I don’t think it is, personally. I know MCP has tradeoffs, and I’ve certainly been a critic of MCP, but I also think it has some virtues. Here’s its primary virtue, in my opinion. If you move outside of the realm of software engineering, the relevant information (context) is not centralized as it is in source code, it resides behind REST API’s. The dream of AI is to ask an agent a question and have it rummage unrestricted through the vast collection of REST API’s you, as a knowledge worker, typically access through web UI’s. You, as a knowledge worker, know that the hardest part of using all of those SaaS web products is logging in. Love it or hate it, MCP offers a realistic path for non-engineers to connect their favorite AI tool to all of those API’s. For the most part, MCP is just the dumb new transport through which AI will end up connecting to REST API’s. OAuth may be a pain, but your marketing folks are probably going to have an easier time with that than setting up a sandboxed virtual machine to allow an agent unfettered access to a lot of command line tools. MCP may have some issues, but it does offer a realistic onramp to connecting all your data sources to an AI. I just don't know what that kind of onboarding looks like for CLI agents. So what’s the solution? I have no idea. But maybe there are stupid ways we can work around the limitations of MCP. I offer you one such stupid example here.

JQ Expression Tool Inputs

First, most MCP tools just return JSON. Too much JSON, as we have already covered, but JSON. Second, we’ve already established that models are trained to be absolute power users of UNIX command line tools. The undisputed champion of concise transformation of JSON on the unix command line is the tool jq. Why not just have our MCP tools accept their transformation as an input. Let me show you what I mean.

Consider again an MCP that provides weather information. We might have a tool that returns the hourly forecast for the next 48 hours. The output of such a call might look something like this,

[
  {
    "time": "2026-03-11T10:00:00Z",
    "summary": "Mostly Clear",
    "icon": "clear-night",
    "temperature": 60.49,
    "apparent_temperature": 59.9,
    "wind_speed": 5.36,
    "wind_bearing": 220,
    "pressure": 1018.11,
    "humidity": 0.94,
    "precipitation_probability": 0,
    "precipitation_intensity": 0,
    "dew_point": 58.24,
    "visibility": 10,
    "cloud_cover": 0.18
  },
  // ... 47 more of these ...
]

Typically, this tool might take no input arguments or it might have a random set of knobs giving limited control of what is returned. What if, instead, it took in a jq expression that allowed the agent to transform the JSON to only what it needs. In the MCP server, the tool’s input schema might look like this,

{
  "description": "Get the hourly weather forecast for the local area for the next 48 hours",
  "inputSchema": {
    "additionalProperties": false,
    "properties": {
      "jq_expression": {
        "description": "A JQ expression to transform the hours property in the result to select only the data that is needed. The context (.) refers to the array contained in the hours property.",
        "type": "string"
      }
    },
    "type": "object"
  },
  "name": "hourly_forecast",
  "outputSchema": {
    "properties": {
      "filtered_hours": {
        "description": "The hours property filtered by the JQ expression"
      },
      "hours": {
        "description": "Hourly weather forecast for the local area for the next 48 hours",
        "items": {
          "properties": {
            "apparent_temperature": { "type": "number" },
            "cloud_cover": { "type": "number" },
            "dew_point": { "type": "number" },
            "humidity": { "type": "number" },
            "icon": { "type": "string" },
            "pressure": { "type": "number" },
            "summary": { "type": "string" },
            "temperature": { "type": "number" },
            "time": { "type": "string" },
            "visibility": { "type": "number" },
            "wind_bearing": { "type": "number" },
            "wind_speed": { "type": "number" }
          },
          "required": [ "time", "summary", "icon", "temperature", "apparent_temperature", "dew_point", "humidity", "wind_speed", "wind_bearing", "visibility", "cloud_cover", "pressure" ],
          "type": "object"
        },
        "type": [ "null", "array" ]
      }
    },
    "type": "object"
  }
}

If that wall of JSON looks unfamiliar, it's how a tool residing in an MCP server is presented to the agent. I have removed some of the details to make it more readable. See the MCP Specification for the full details.

In theory, this gives our agents a knob they can use to filter the data to only what they need, the same kind of knobs that Claude so masterfully deploys when it’s Gibberdibbling… in the terminal. For example, if the agent needs only the temperature for the next 24 hours, it can avoid having the large blob of JSON in the context window by calling the houly_forecast tool like this,

hourly_forecast({jq_expression: “[ .[0:24][] | .temperature ]”})

Returning a nice, concise array of temperatures.

[ 39.06, 37.89, 36.72, 37.72, 39.24, 41.4, 43.34, 44.46, 46.18, 47.32, 47.89, 48.16, 47.41, 46.72, 44.98, 43.63, 43, 42.66, 41.41, 40.46, 39.51, 38.43, 37.33, 35.51 ]

Does this work?

Yes. I think so. Honestly, I don’t have enough data to know how reliable this is, but it is passing the vibes check. And to my surprise, it’s even passing the vibes check on small open weights models. Here’s gpt-oss:20b making use of the jq expressions.

Better models, as you would expect, do even better. Not only do they tend to get the expression right on the first iteration, but they correct mistakes far faster than the smaller models. That’s not surprising. I can’t say definitively if this “works” for production agents, but it has been working well enough for me that I’m inclined to explore it further.

Other Alternatives

As I said earlier, I don't want to throw out MCP servers right now. Most AI tools can connect to them. Most companies plan to ship one. The protocol and infrastructure is far from ideal, but it currently seems to be the most practical way to give an AI agent access to most of your important data. I can configure Claude with MCP servers and there is a reasonable set of steps to authenticate with OAuth so that the MCP server knows who I am and what data I can access. Throwing all of that out in favor of command line tools would, at least in the short term, lock out most users. So even if the JQ expressions strategy that I describe above doesn’t work out, I think it’s worth continuing to try to make MCP servers work. For instance, the Anthropic article that I posted earlier, …, describes programmatic tool calling where they use a python runtime to call tools. Claude writes python that calls tools and transforms the data as needed to satisfy the goal. This is a cool idea, but it does require a full runtime. In the case of Antropic, they are using python. I have also done some prototypes that do this with Javascript. Allowing agents to run arbitrary code introduces a lot more complexity, which is why I’m trying to start with a very limited runtime, jq.

Code on Github

Reading with LLMs

2025-10-13T00:00:00Z

TL;DR;

This is a demo of using LLMs to summarize books to the desired reading length.

Like any everyone else with a keyboard, I spend a lot of my time these days using Large Language Models. It is both exciting, in that this stuff is new and the applications seem endless and unexplored, but it is also frustrating, in that most real uses of AI have received lukewarm receiptions from users. Some studies even suggest that consumers are averse to buying products that include AI in their descriptions. It seems fair to say that we're still just figuring out what this stuff is good for.

This post is me sharing one of my own explorations into what AI might be good for. The demonstration is below, jump ahead if you'd like, but I'll try to keep the background short. In the last few years, I have had very little time for reading. In those rare moments when I expect to have some extended quiet time for reading a book, I struggle to pick the book. Since it will likely be the only book that I read for a while, the stakes seem higher and so I spend far too much time researching. There was also this dark, but memorable, Books Before You Die Calculator that still haunts me. I've come to realize that there are books that I would like to read but there are far more books that I'd like to understand as if I had read them, but without the time commitment. Perhaps that former category of "reading" is something appropropriate for AI?

Tangentially, I also tend to buy audiobooks for long road trips and I've had some success getting the gist of a book by asking ChatGPT or Gemini for a directed summary. For instance, I recently asked this of Gemini and the results were enough to convince me to pick the book for an upcoming road trip.

I am considering reading Burning Down the House: How Libertarian Philosophy Was Corrupted by Delusion and Greed by Andrew Koppelman. Can you please list the chapters and tell me what I'm likely to take away from each chapter?

It only hallucinated one of the chapters and even for that chapter, the content that it summarized was actually in the book. For technologists who spend a lot of time with AI, summarization of text doesn't seem very interesting, but it is a task that LLMs seem to do quite well. The demo below came about as I was wondering if LLMs could provide me with a way to "read" a book without the time commitment of actually reading the book. You start with the original text but if that's too much of a commiment, you ask the LLM to reduce the text to a more manageable size.

Before you start clicking, let me give you some caveats. This is not a live demo. I cannot afford to run a live production model on my personal site. I generated the summaries in advance at a few target lengths. Hopefully you can imagine how this might be improved if a live model were available. Given that the summaries are pre-computed, you might wonder why there are incremental text transformations between the summarization levels. The answer is that I've become quite fond of the aesthetic of LLMs streaming in tokens and I was thinking of ways that might be preserved when text is being mutated, rather than produced. Those transformations are my take on preserving the streaming token aesthetic when a block of text is being updated.

With the background and caveats out of the way, here's the demo.

Bouncing Ball (AI Updated)

2025-07-13T00:00:00Z

Way back in February 2006, I wrote a post where I shared a really simple way to model a bouncing ball that produced somewhat realistic rotations when the ball collided with a surface. It was all based on a 1969 American Journal of Physics article that I somehow stumbled over while doing the first week exercise in Neil Gershenfeld's famous Nature of Mathematical Modeling course at MIT. Like so many posts from that time, I was desperately trying to turn some of my school work into content on my site to keep it alive even though I was blindingly busy. It's often hard for me to re-read some of those posts because they seem so silly now but I keep them up out of principle. The post in question here is nearing 2 decades of being up on the internet. This post, though, has another distinction among all the other posts on my site. It has the highest JavaScript error rate, which I can easily see with Fullstory. The reason the error rate is so high is that the original post contained a bouncing ball simulation that was written in Java and published as a Java Applet. Many of you are not even familiar with this term, “Java Applet”. There was a time when we were all convinced that the future of programming from web pages to toasters was Java and one part of that vision was the ability to stick an element on your web page that ran a Java Virtual Machine. It even included the ability to draw things using Java's graphics libraries. It did not, as some expected, pave the way to an all Java future but it did create an impressive number of security vulnerabilities that was eclipsed only by the king of insecurity, Adobe/Macromedia Flash. This is why browsers no longer support Java Applets and they haven't supported them since around 2015. You know what happens to most pages that still have functionality in a Java Applet? They throw an error. I still get regular but slow stream of curious people arriving on that Bouncing Ball blog post, eager to see it all in action, only to find it no longer works. And on the other side, I glance at my site stats every few months to find that page still topping the list of Error Pages.

I have, for many years, wanted to take some time to remove the Applet and replace the ball simulation with something more modern, like an HTML Canvas element, but I never have taken the time to do it. During the few times when I found enough motivation to open the old Java code to assess how long it would take, I quickly retreated because it's 20 year old Java code written by a sleep deprived graduate student with very little professional software development experience. In industry parlance, “the code sucks”. Earlier today, though, I decided to do the thing that everyone else has been doing when they want some code created, but there's little reason to invest in making the code “not suck”. I asked Cursor to port it from Java to Typescript. I don't think this qualifies as vibe coding because while I did defer a coding task to an LLM, I ultimately tried to verify that the code does what I intended. To get started, I typed the following prompt,

In 2006, I wrote a Java Applet that simulated a bouncing ball using a simple model that simulated angular velocity changes on collisions. The code for that applet is in the @/java directory. I want to recreate that simulation in an HTML canvas in the vite app. The entry point for the vite app is @index.html . Can you port the Java into Typescript?

Cursor jabbered at me in the chat window for a bit and I dutifully ignored all the text, skimmed the code, and quickly accepted each of it's suggestions. I followed up a few times to ask it to add pieces that were missing from the original, including this prompt that I saved.

In the original, the ball was rendered with 2 images. @lotto-hi.png and @lotto-lo.png. @lotto-lo.png was the image of the ball's surface and was meant to rotate. @lotto-hi.png was meant to simulate ambient light and did not rotate with the ball. Can you render the ball using these images? You should make copies of them in the @/src directory.

There were some other prompts where I asked it to keep fixing various little issues or add missing elements. But after a few minutes, I did have the bouncing ball simulation running as a vite app using a canvas element. The code still sucked since the LLM mostly just ported over the Java code, but it worked and I had spent very little time on the porting effort. What you see below is the result. Well, that's mostly the result. I did fix several things, by hand, because I don't want this page to become my new leading error page. I can now update the original post to point here and be done with a decade of Java Applet errors.

The Bouncing Ball

Equations

I have to believe that many of the people who land on the old page are getting there because they are attempting to create their own bouncing ball simulation. In 2006, it was pretty hard to put math equations in web pages and so I embedded them in the page as images. Here are those same equations in a more readable form,

$\\begin{align} v*{x2} &= \\frac{(1 - \\alpha e_x) v*{x1} + (\\alpha - e*x) R \\omega_1}{(1 + \\alpha)}, \\\\ v*{y2} &= - e*y v*{y1}, \\\\ \\omega*2 &= \\frac{(1 - \\alpha e_x) \\omega_1 + (\\alpha - e_x) R v*{y1}}{(1 + \\alpha) R} \\end{align}$

$\alpha$ represents rotational inertia (2/5 is a typical value).
$v_1$ is the velocity of the ball before the collision.
$\omega_1$ is the angular velocity of the ball before the collision.
$R$ is the radius of the ball.
$e_x$ and $e_y$ are the coefficients of restitution.
$v_2$ and $\omega_2$ are the velocity and angular velocity of the ball after the collision.

Finally, all the links in the original post are now broken. Here are the updated links,

Closing Thoughts

Did using AI save me a lot of time? Not really. The task of porting the code would not have taken that long and the code that the LLM produced was pretty hard to integrate and change. I can't really blame the model, I gave it garbage code as input. However, I really had no interest in recreating this project by hand. I viewed it as tedious and using Cursor made it fun, which I really do appreciate. With AI coding, I think we probably spend too much time arguing about 100x productivity promises and not enough about enjoying the experience. Using AI coding tools is a lot of fun. If you're interested in seeing what Cursor produced, you can find that in Github.

PostgreSQL Slow Cascading Deletes

2025-06-29T00:00:00Z

This is less a thing I recently learned than one that I re-learned multiple times through real-world experience. Imagine that you are migrating a reasonable sized dataset from a document store to PostgreSQL. You have enabled dual writes and you are progressing slowly through a backfill process when, sadly, latency begins to spike and you find that all the slow queries are DELETE statements and PostgrSQL is consuming quite a bit of CPU trying to complete these deletes. When you look at the schema, you see that the deletes are associated with a foreign key constaint that specifies ON DELETE CASCADE. You might be tempted, as many on stack overflow have done, to conclude generally that "DELETE CASCADE in PostgreSQL extremely slow". That's not strickly true, though. Let's take a look at an example and explain why this happens.

First, let's create a couple of tables that have the kind of foreign key constaint that we're talking about.

CREATE TABLE IF NOT EXISTS people (
  id INTEGER PRIMARY KEY,
  name VARCHAR(255) NOT NULL);
CREATE TABLE IF NOT EXISTS phone_numbers (
  id INTEGER PRIMARY KEY,
  person_id INTEGER NOT NULL REFERENCES people(id)
    ON DELETE CASCADE ON UPDATE CASCADE,
  number VARCHAR(255) NOT NULL);

Now, let's run a little experiment where we insert N rows into the people table and N rows into the phone_numbers table. Then, we'll simply delete all the rows in the people table (which will also delete all the rows in the phone_numbers table) using the following statement,

DELETE FROM people;

We'll do this for increasing values of N and we'll also do the same thing in MySQL to see if PostgreSQL deserves the reputation for having slow cascading deletes.

That clearly shows that there is some merit to the claim that PostgteSQL has slow cascading deletes. MySQL is able to delete all 25,000 people rows along with their associated phone numbers in 174ms. PostgreSQL, on the other hand, takes over 14 seconds. Such a massive difference in performance is a good indicator that this is not merely a matter of PostgreSQL being slower than MySQL. Let's look and see what indexes were created in PostgreSQL when we built our tables earlier.

# SELECT tablename, indexname
FROM pg_indexes
WHERE schemaname = 'public';
   tablename   |     indexname      
---------------+--------------------
 people        | people_pkey
 phone_numbers | phone_numbers_pkey
(2 rows)

The only indexes that we have are the primary key indexes for each of the tables. When we delete a row from people, we have to be able to find the corresponding rows in phone_numbers that have the same people_id in order to delete them as well. As you can see, though, there is no index on the person_id column in phone_numbers. Our cascading delete on the phone_numbers table requires a full table scan. Does this suggest that MySQL builds indexes automatically that PostgreSQL does not? Let's see.

mysql> SELECT table_name, index_name
FROM INFORMATION_SCHEMA.STATISTICS
WHERE table_name IN ('people', 'phone_numbers');
+---------------+------------+
| TABLE_NAME    | INDEX_NAME |
+---------------+------------+
| people        | PRIMARY    |
| phone_numbers | PRIMARY    |
| phone_numbers | person_id  |
+---------------+------------+
3 rows in set (0.004 sec)

It appears so. MySQL not only has an index on the primary key of phone_numbers but it has also added an index on person_id. This explains why PostgreSQL is so slow relative to MySQL. Let's add that index in PostgrSQL to confirm.

CREATE INDEX phone_numbers_number_idx ON phone_numbers (person_id);

As expected, PostgreSQL performance, when you add the missing index, is now on par (a bit better, actually) than MySQL.

Is this on purpose?

It seems so. Let's look at the PostgreSQL documentation first.

Since a DELETE of a row from the referenced table or an UPDATE of a referenced column will require a scan of the referencing table for rows matching the old value, it is often a good idea to index the referencing columns too. … declaration of a foreign key constraint does not automatically create an index on the referencing columns. PostgreSQL 17.5: 5.5.5. Foreign Keys

While the documentation doesn't explicitly say why PostgreSQL decided not to create an index automatically in this case, the section of their documentation on indexes suggests using them "sensibly" and leaving their design to the application developer seems consistent with this principle.

Indexes are a common way to enhance database performance. An index allows the database server to find and retrieve specific rows much faster than it could do without an index. But indexes also add overhead to the database system as a whole, so they should be used sensibly. PostgreSQL 17.5: 11. Indexes

And if we look at the documentation for MySQL, we see that while they do create these indexes automatically, there seems to be more complexity than is apparent in our trivial example.

MySQL requires indexes on foreign keys and referenced keys so that foreign key checks can be fast and not require a table scan. In the referencing table, there must be an index where the foreign key columns are listed as the first columns in the same order. Such an index is created on the referencing table automatically if it does not exist. This index might be silently dropped later if you create another index that can be used to enforce the foreign key constraint. index_name, if given, is used as described previously. MySQL 8.4: 15.1.20.5 FOREIGN KEY Constraints

As you can see, MySQL will create an index on the referencing table if one doesn't already exist but it can use another existing index if it contains the neccessary keys. It also suggests that if an index is later created that can be used to enforce the foresign key containt, the automatically created index will be dropped. I can understand why PostgreSQL would want to avoid this complexity and ultimately defer to the developer to decide on a "sensible" approach.

Conclusion

Modern relational databases are complex and I think there is a tendency to treat them as if they are somewhat magical. But despite the many cool things they can do, the basic data structures used to store and retrieve data haven't really changed in decades and they have tradeoffs. Every index you add to your database will create additional storage and will add write overhead. However, you have to make sure that you have indexes to support the read patterns needed by your application. These are design tradeoffs that just cannot be made in the database itself. It's admirable that MySQL has attempted to address this foot gun. At the same time, you can understand why PostgreSQL would want to avoid silently creating indexes that were never requested.

After dealing with relational databases for many years as a product-level engineer, I've become pretty suspicious of any kind of implicit magic at the storage level. I've also had all of my initial hopes that code reviews would be effective in reviewing schema changes crushed repeatedly. The one thing that has saved me time and time again, though, has been runtime logging of slow or unindexed queries, via MySQL's Slow Query Log or through the pg_stat_statements extension in PostgreSQL.

0.1 + 0.2 != 0.3

2024-07-16T00:00:00Z

Recently someone shared a link to a Javascript quiz that asks various questions about Javascript's infamous quirks. It was a fresh remix on a greatest hit that we've all been jamming to since the early 2000's. And within that quiz was a question that seems to appear in most of these quizzes about Javascript:

What will be the output of this code?

console.log(0.1 + 0.2 == 0.3);

This is a real fun question and it never ceases to amaze at least one React developer each time it's shared in slack (React, after all, is just the new jQuery, right? That's a joke, guys). This question, though, doesn't have a thing in the world to do with Javascript and, like a crumugeon, I've been well-actuallying this question since these quizzes first started appearing. In case you haven't already guessed, in Javasript (node) this produces the following output …

$ node math.js
false

Not a Javascript thing at all

Like I said, this is not a problem with Javascript. To prove it, here's similar code in all your favorite (and least favorite) languages. Each of these will print false or an equivalent.

Python

print(0.1 + 0.2 == 0.3)
# False

Ruby

puts(0.1 + 0.2 == 0.3)
// false

PHP

<?php echo (0.1 + 0.2 == 0.3) ? 'true' : 'false';
// false

Java

class Main {
  public static void main(String[] args) {
    System.out.println(0.1 + 0.2 == 0.3);
  }
}
// false

#include <stdio.h>
int main() {
  printf("%d\n", 0.1 + 0.2 == 0.3);
}
// 0

package main
import "fmt"
func main() {
  a := 0.1 // otherwise the compiler will use constant folding
  fmt.Println(a+0.2 == 0.3)
}
// false

Rust

fn main() {
  println!("{}", 0.1 + 0.2 == 0.3)
}
// false

As you can see, while this question/quirk is regularly attributed to Javascript, this is not behavior that is unique to Javascript. In fact, you can see this behavior in any language that uses IEEE 754 to represent floating point values.

What's going on here?

First, I'm going to try my best to avoid a lot of IEEE 754 details in this post. If you want exhaustive coverage, I highly recommend Computer Systems: A Programmer's Perspective [Section 2.4]. For our purposes, a double precision floating point value (which each of the language above uses) consists of 53 bits¹ for the fractional part (F) of the number and 11 bits for the exponent (E). There is also a sign bit, but we can ignore that entirely here since everything is positive. So we can think of floating point numbers as being scientific notation using base 2 instead of base 10.

$F \times 2^{E}$

If we look at how 0.1 is represented in binary, we end up with this:

0.1₁₀ = 1.100110011001…10011010₂ × 2^-4

You will notice that the fractional part is repeating. In fact, much like ⅓ in base 10, the fractional part repeats indefinitely (0.3333…). In order to fit this infinitely repreating fraction into 53 bits, we have to round it. In this case, we round up since the next bit is 1 and not 0 (1001). That's how we end up with 1010 at the end, instead of 1001. You may be surprised that 0.2 has the exact same fractional component as 0.1 but 0.2 is 2 × 0.1, so it makes a lot of sense that we would just change the exponent.

0.2₁₀ = 1.100110011001…10011010₂ × 2^-3

And finally, here's 0.3. You'll notice that the repeating part of 0.3 is a 0011 (3) and so it is actually rounded down.

0.3₁₀ = 1.001100110011…00110011₂ × 2^-2

0.1 + 0.2

Now, let's add 0.1 and 0.2 together and see what happens. To add 0.1 and 0.2, we first have to rewrite the smaller number so that it has the same exponent as the larger number. Then we can simply add the fractional parts using base 2 addition. Here's what that looks like:

0.1100110011001 … 10011010 × 2^-3 + 1.100110011001 … 10011010 × 2^-3 = 10.011001100110 … 011001110 × 2^-3

Because of the shifting and the carry, we now have 55 bits so we're going to need to round away 2 bits. However, the bits that we need to discard are 10, so we have to round up, giving us.

1.001100110011…0011001110₂ × 2^-2

0.1₁₀ + 0.2₁₀ = 1.001100110011…00110100₂ × 2^-2

This is the value of about 0.30000000000000004 which we would have seen if we printed the result of 0.1 + 0.2. Obviously, this is ever so slightly different from 0.3.

1. The fractional part in a double precision floating point number is actually only 52 bits. In normalized form, there is an implicit 1 in front of the fractional part. So while it is 52 bits in memory, it represents a 53 bit value. [back]

A Thing I Learned: counting bits

2023-07-24T00:00:00Z

How many bits are set to one in a bit string? It’s a problem that pops up in a number of areas: bitsets, cryptography, error correction, and most of all interview questions. This problem is also known by many names: hamming weight, population count, popcount, sideways sum just to name a few. Almost any software engineer can write a trivial implementation using bit shifts in minutes, yet this problem has a long history and, even today, you will find a variety of different implementations for this seemingly simple problem. I recently revisited this problem in the context of writing a simple bitset and found myself falling down a rabbit hole when I tried to answer the question: how is std::popcount implemented in C++20?

There are many ways to count bits

Let’s start with the most straight-forward approach. This is the solution you are most likely to get if you were to ask a software engineer in an interview to count the number of bits set to one in a 64-bit unsigned integer. You simply shift the value right and accumulate the sum of each right most bit until the value is zero.

int count_bits(uint64_t v) {
    int count = 0;
    for (; v != 0; v >>= 1) {
        count += v & 1;
    }
    return count;
}

There is also a similar, but more clever, version commonly known as Brian Kernighan’s Algorithm. Kernighan is credited with this strategy because of this exercise that appears in the 2nd Edition of The C Programming Language.

Exercise 2-9. In a 2’s complement number system, x &= (x - 1) deletes the rightmost 1-bit in x. Explain why. Use this observation to write a faster version of bitcount.

The approach uses the expression above and counts the number of rightmost bits that are deleted until x is zero (no more 1-bits). So unlike the earlier approach, this one executes on the order of the number of 1-bits, not the number of bits in the word.

int count_bits(uint64_t v) {
    int count = 0;
    for (; v != 0; count++) {
        v &= v - 1;
    }
    return count;
}

The most widely used implementation, though, seems to be the approach that was included in the 2nd edition of Hacker’s Delight by Henry Warren. This strategy is kind of hard to explain but the book includes an explanation and a proof. This is what you will find in Go’s math/bits implementation of OnesCount64. Here’s what the code looks like in the Go source tree currently (with some of the comments removed).

const m0 = 0x5555555555555555 // 01010101 ...
const m1 = 0x3333333333333333 // 00110011 ...
const m2 = 0x0f0f0f0f0f0f0f0f // 00001111 ...
const m3 = 0x00ff00ff00ff00ff // etc.
const m4 = 0x0000ffff0000ffff
// OnesCount64 returns the number of one bits ("population count") in x.
func OnesCount64(x uint64) int {
	// Implementation: Parallel summing of adjacent bits.
	// See "Hacker's Delight", Chap. 5: Counting Bits.
	// The following pattern shows the general approach:
	//
	//   x = x>>1&(m0&m) + x&(m0&m)
	//   x = x>>2&(m1&m) + x&(m1&m)
	//   x = x>>4&(m2&m) + x&(m2&m)
	//   x = x>>8&(m3&m) + x&(m3&m)
	//   x = x>>16&(m4&m) + x&(m4&m)
	//   x = x>>32&(m5&m) + x&(m5&m)
	//   return int(x)
	const m = 1<<64 - 1
	x = x>>1&(m0&m) + x&(m0&m)
	x = x>>2&(m1&m) + x&(m1&m)
	x = (x>>4 + x) & (m2 & m)
	x += x >> 8
	x += x >> 16
	x += x >> 32
	return int(x) & (1<<7 - 1)
}

There is an instruction for this

All those clever approaches above, it turns out we don’t really need any of them now. As I was working on my bitset implementation, I used std::popcount from C++20 and looking at the generated code on my M1 Mac, the assembly is surprisingly concise.

<count_bits>:
  fmov d0, x0
  cnt.8b v0, v0
  uaddlv.8b h0, v0
  fmov w0, s0
  ret

The CNT instruction on arm64 is a population count instruction. It is a little strange in that it’s a SIMD instruction so you have to move the value from a general register into a vector register and also sum across the vector register. But none of the clever tricks from the long history of clever tricks are needed on ARM processors. And indeed, compilers on my Mac M1 are very happy to use the CNT instruction. This swift code, for instance, uses CNT,

func countCounts(_ x: UInt64) -> Int {
    x.nonzeroBitCount
}

As does this rust code,

fn count_bits(v: u64) -> u32 {
    v.count_ones()
}

But what about x86-64? It also has a POPCNT instruction and it was introduced a long time ago in SSE4.2 (2008). I have a number of x86-64 chips in my house and they support POPCNT (You can usually check with the linux lscpu command). For a more exhaustive look at the history of the population count CPU instruction, I suggest Vaibhav Sagar’s post You Won’t Believe This One Weird CPU Instruction! which also includes a number of historical links.

Here's another thing that will bake your noodle. If you were to compile the Kernighan version of count_bits above with clang or gcc, you will very likely find that it also uses the CNT instruction on arm64. These compilers are smart enough to recognize a few common implementations of population count and replace your code with its own builtin version that is optimized for the target CPU.

As it turns out, counting the number of 1-bits in a bit string isn't as simple as it seems. if you want to read more, I recommend the following links:

Wikipedia: Hamming Weight
Vaibhav Sagar’s You Won’t Believe This One Weird CPU Instruction!
Daniel Lemire's The surprising cleverness of modern compilers

A Thing I Learned: constexpr & consteval

2023-07-15T00:00:00Z

A Thing I Learned is intended to be an ongoing series where I teach myself to write narrowly about some small thing that I have recently discovered that may also be of interest to others. In this first post in the series, I’m going to talk about a couple of new specifiers that have been added to modern C++, constexpr and consteval and how I recently had a revelation that their compile time capabilities were beyond what I originally understood.

While I never write C++ professionally these days, I do have fond memories of previous professional projects where C++ was the primary language and I harbor just enough nostalgia to try to keep up with some of the newer features added to the language. Recently, I decided to see if I had kept up with the new features enough to write a sudoku solver. This is something I often write when I’m attempting to learn a new language. Since I’m already familiar with my own solution, I can focus on learning the appropriate way to write it in the given language. Like many sudoku solvers in existence, mine derives from Peter Norvig’s famous post Solving Every Sudoku Puzzle. As Norvig explains, we can solve sudoku puzzles efficiently by using constraint propagation and then doing a backtracking search on what remains. If you are interested in those details, you really should read the original. I’m only going to talk about a very specific part of the solution and those are the constraints that are implicit in the sudoku puzzles. As Norvig describes it, every square has exactly 3 units and 20 peers. As an example, if square 0 is known to have the value of 1, then none of its other peers can have that value. The list of peers for each square is the data we need to do the constraint propagation. The peers are higlighted in yellow for each square in the puzzle below.

Typically when I write this sudoku solver, I build the list of peers as a static array computed once during runtime. The code for that in C++ might look something like this.

size_t quad_of(size_t ix) {
    return (ix % 9) / 3 + 3 * (ix / 27);
}

size_t col_of(size_t ix) {
    return ix % 9;
}

size_t row_of(size_t ix) {
    return ix / 9;
}

auto get_peers() {
    std::vector<size_t> peers;
    peers.reserve(81 * 20);

    for (auto i = 0; i < 81; i++) {
        auto row = row_of(i);
        auto col = col_of(i);
        auto quad = quad_of(i);
        for (auto j = 0; j < 81; j++) {
            if (i == j) {
                continue;
            }

            if (row == row_of(j) || col == col_of(j) || quad == quad_of(j)) {
                peers.push_back(j);
            }
        }
    }
    return peers;
};

// ...

bool Puzzle::assign(int ix, int value) {
    static auto peers = get_peers();
    // ...
}

Each time I implement this, I’m tempted to generate the peers in advance and embed it in the code as an array literal. That’s nice in that it ensures the data is embedded in a read-only segment of the executable but to do that I would end up writing a small script to generate that array literal and then I’m left wondering if I should keep the code generation script or delete it. Also the source ends up cluttered with an unseemly 1,620 element array literal. Is it possible, I wondered, to build this array during compile time using something like constexpr? Turns out the answer is yes and it’s all pretty straightforward. First, though, let me give you a simple description of what I mean by constexpr and consteval.

What is a constexpr and a consteval?

Despite what Google Bard might tell you, constexpr was introduced in C++11, not C++17, and has been steadily improved since. When you add the constexpr specifier to a function or a value the compiler ensures that that function or value can be evaluated at compile time.

The consteval is a related specifier, introduced in C++20, that directs the compiler to ensure that the value or function is evaluated at compile time. As you might expect, a function that is specified with consteval can call one that is specified with constexpr, but not the other way around, which would be a compile time error.

So in theory, if we can create a consteval function that builds the table of peers we can evaluate that function entirely at compile time. Since it’s constexpr function, we know that any calls to it must be evaluated at compile time.

Computing the peer table at compile time

First, we know that our consteval get_peers function is going to need to call all constexpr functions, so there is the trivial matter of adding the constexpr specifier to row_of, col_of and quad_of. That’s no problem since the expressions in each of those functions are pretty trivial to do at compile time.

constexpr size_t quad_of(size_t ix) {
    return (ix % 9) / 3 + 3 * (ix / 27);
}

constexpr size_t col_of(size_t ix) {
    return ix % 9;
}

constexpr size_t row_of(size_t ix) {
    return ix / 9;
}

There is one wrinkle, though, with our current get_peers function. std::vector uses dynamic memory allocations, which cannot be done at compile time. Apparently there are ways to use std::vector in a very limited manner inside of a constexpr function. Doesn’t matter, though, we don’t need the dynamic nature of a std::vector as we know the exact size of the table we’re building. We can easily switch to std::array with a statically declared size.

consteval auto get_peers() {
    std::array<size_t, 81 * 20> peers = {};
    auto iter = peers.begin();
    for (auto i = 0; i < 81; i++) {
        auto row = row_of(i);
        auto col = col_of(i);
        auto quad = quad_of(i);
        for (auto j = 0; j < 81; j++) {
            if (i == j) {
                continue;
            }

            if (row == row_of(j) || col == col_of(j) || quad == quad_of(j)) {
                *iter = j;
                iter++;
            }
        }
    }
    return peers;
}

In the function where we need the table of peers, now we can just make a local static.

bool Puzzle::assign(int ix, int value) {
    static constexpr auto peers = get_peers();
    // ...
}

Does it work?

According to the C++20 specification, we should feel confident at this point that the table of 1,620 should have been generated at compile time and should be embedded somewhere in our executable. Computers, though, are liars. Just a few minutes ago, Google Bard told me that both constexpr and consteval were introduced as part of C++17. I’d rather verify this myself. If this table really is being computed at compile time, I’d expect to find the full contents of the table in one of the read only sections of the executable. I’m doing this on a Mac M1 so my hunch is that my table is going to be in the __const section (within the __TEXT segment) of the Mach-O object. Let’s see.

$ objdump -sj __const puzzle.o

puzzle.o:	file format mach-o arm64
Contents of section __TEXT,__const:
0fb0 01000000 00000000 02000000 00000000  ................
0fc0 03000000 00000000 04000000 00000000  ................
0fd0 05000000 00000000 06000000 00000000  ................
0fe0 07000000 00000000 08000000 00000000  ................
0ff0 09000000 00000000 0a000000 00000000  ................
1000 0b000000 00000000 12000000 00000000  ................
1010 13000000 00000000 14000000 00000000  ................
1020 1b000000 00000000 24000000 00000000  ........$.......
1030 2d000000 00000000 36000000 00000000  -.......6.......
1040 3f000000 00000000 48000000 00000000  ?.......H.......
1050 00000000 00000000 02000000 00000000  ................
1060 03000000 00000000 04000000 00000000  ................
1070 05000000 00000000 06000000 00000000  ................
1080 07000000 00000000 08000000 00000000  ................
1090 09000000 00000000 0a000000 00000000  ................
10a0 0b000000 00000000 12000000 00000000  ................
10b0 13000000 00000000 14000000 00000000  ................
...

I guess it did work, after all. That’s clearly the table of peers, albeit in 64 bit little endian values. Should you, for some strange reason, have an interest in the sudoku solver that prompted this exploration, you can find it in my github. And if you are interested in a pretty solid intro to modern C++, I enjoyed Scott Meyer’s Effective Modern C++.

What time is March 14, 2021 2:00 AM?

2022-06-27T00:00:00Z

This post is just some fascinating (or quite boring depending on your perspective) minutia that arises in the handling of time and timezones in programming. This deals with a little corner that exists in almost every time library yet I’ve never had much reason to explore it. I had just assumed that all major programming environments would return a consistent answer to this question. Surprisingly, the answers are a bit less consistent than I expected.

What special about March 14, 2021 2:00?

That’s the exact time in 2021 for the vast majority of Americans when the dreaded Daylight Savings Time began. I’m on the east coast (my tz database timezone is America/New_York) and so, for me, this was the time that my timezone transitioned from Eastern Standard Time (5 hours behind GMT) to Eastern Daylight Time (4 hours behind GMT). Because of this transition, there is no 2am on March 14, 2021. If you were watching a microwave clock (assuming that it adjusted for DST) early in the morning on March 14, it would transition from 1:59AM to 3:00AM. In local time, there is no such thing as March 14, 2021 2:00AM. The diagram below is another illustration of why 2:00 never happens on March 14.

But what happens if I use the time library in my favorite programming language and try to construct that time? This adventure began for me in Go, which is a programming language I work in often. I was building test cases for a function that operated on time.Time values and, unsurprisingly, I wanted to include test inputs that verified that this function acted reasonably during daylight savings time transitions. So I typed the following code not really knowing what would be returned.

What it returns, when using Go 1.18.3, is 2021-03-14T01:00:00-05:00. That result was a little surprising to me, at first, but it does make sense. This clearly interprets the input as indicating 2am EDT (the time zone after the transition) but that time would be presented in EST (the time before the transition) for America/New_York and so the result is 1am EST. I took a look at the documentation for Go’s time package and, to my surprise, this case is covered in the documentation. As it turns out, not all time libraries bother to cover this behavior in documentation. Here's what Go promises in this case,

A daylight savings time transition skips or repeats times. For example, in the United States, March 13, 2011 2:15am never occurred, while November 6, 2011 1:15am occurred twice. In such cases, the choice of time zone, and therefore the time, is not well-defined. Date returns a time that is correct in one of the two zones involved in the transition, but it does not guarantee which.

So Go is light on its guarantees, you’ll get a time interpreted in one of the two time zones involved in the transition, you can’t expect anything more than that. Given Go’s reluctance to offer more predictable guarantees, I started exploring other languages and their associated time libraries. Would there be consistency here? Would the documentation describe the expected behavior of this case? I took a look at just a few of the most popular languages to see.

JavaScript

JavaScript is not only the world’s most popular programming language but it’s behavior is pretty tightly specified in the EcmaScript Language Specification including the Date Object. I tried it next.

It returns 2021-03-14T03:00:00-04:00, which is different from what got with Go. JavaScript seems to have interpreted the input as indicating 2am EST (the time zone before the transition). In America/New_York that time would fall into EDT and so the result is 3am EDT. For whatever reason, this is the result I had anticipated with Go. Given that JavaScript has a very complete specification, I would expect this behavior to be specified, and it is in Section 21.4.1.7.

When t_local represents local time repeating multiple times at a negative time zone transition (e.g. when the daylight saving time ends or the time zone offset is decreased due to a time zone rule change) or skipped local time at a positive time zone transitions (e.g. when the daylight saving time starts or the time zone offset is increased due to a time zone rule change), t_local must be interpreted using the time zone offset before the transition.

JavaScript makes more precise guarantees than Go in that it requires that the time be interpreted using the timezone before the transition. I continued through a few more languages.

PHP

Another very popular language that receives a lot of hate, what does it do?

2021-03-14T03:00:00-04:00 It’s consistent with JavaScript. It interpreted the time using the zone before the transition. Is this described in the documentation? Not that I could find. The PHP documentation tends to be pretty complete but I did not find this behavior described anywhere in the section about Date and Time Related Extensions. Moving on,

Java

2021-03-14T03:00:00-04:00 It is also consistent with JavaScript and PHP. Is it covered in documentation? Yes and right at the top of the documentation for the ZonedDateTime class.

For Overlaps, the general strategy is that if the local date-time falls in the middle of an Overlap, then the previous offset will be retained. If there is no previous offset, or the previous offset is invalid, then the earlier offset is used, typically "summer" time.. Two additional methods, withEarlierOffsetAtOverlap() and withLaterOffsetAtOverlap(), help manage the case of an overlap.

ZonedDateTime even exposes API to get a time using either the earlier or later offset.

Python

2021-03-14T02:00:00-04:00 That’s surprising and definitely inconsistent with other languages. What I find strange about this result is that it's not even in the requested time zone. It seems to have interpreted the time in EDT (the timezone at the end of the transition) and comes up with the same UTC time as Go (6am UTC). However, that time in America/New_York should be in EST and not EDT. And what about documentation? Yes, but you have to work hard to find it. Python’s datetime library does not provided a tzinfo implementation that handles DST transitions so one has to look to the dateutil.tz module for that and also for the appropriate documentation. The behavior is described in the documentation for the resolve_imaginary function.

This function assumes that an imaginary datetime represents what the wall time would be in a zone had the offset transition not occurred, so it will always fall forward by the transition’s change in offset.

What does that mean? I don’t have a clue and I can’t explain Python’s result after having read it. One more …

Ruby

Same answer as JavaScript, Java and PHP. As far as documentation, I could not find anything in the documentation for ActiveSupport::TimeZone. Like Python, proper handling of tz database names (i.e. America/New_York) doesn’t seem to be in the standard library.

Summary

Here's a summary of the results from each of the languages I tried. Python and Go stand out, of course, and Python stands out most of all since the result is not even in the requested time zone.

What about Nov 7, 2021 at 1am?

There is, of course, the other transition that happens each year when we transition from daylight savings time back to standard time. In this case, though, the time happens twice. If you were staring at your microwave at 1:59am on Nov 7, 2021 (assuming again that you have a fancy microwave that adjusts automatically), it would transition to 1:00am. An hour before that it would have also shown 1:00am. For the sake of brevity, I’ll just summarize. For this transition, all the languages come up with the same answer, strangely.

Fascinating to some, including me, but I’m sure boring to others. I did not, however, expect to find differences in popular programming languages. I should also add that I intended to include rust in this list, but rust’s chrono crate rejects ambiguous times, either by panicking or by returning the Option type of None. I quite like that approach, honestly. I guess the only real take-away here is to avoid these ambiguities that exist in the date time libraries of most programming languages. Otherise, you may end up having a bad time.

Sleep Numbers

2019-12-24T00:00:00Z

About 5 or 6 years ago, I decided to form better sleep habits. I had spent too much of the previous decade in sleep deprivation and even when I was getting sufficient quantities of sleep, my bed times were wildly inconsistent. In graduate school, I had made such a habit of neglecting sleep that my memories of those years are fragmented. I would like to say that that having kids is what prompted me to give up my unhealthy sleep habits, but even when we had three kids, you would still often find me up well past midnight on weeknights. In some ways, having kids made things worse since the night time hours were relatively quiet. I liked having uninterrupted time to focus on projects and 10pm until 4am were hard to beat. Eventually, though, general concern for my health prevailed and I adopted a reasonable bedtime and committed myself to no less than 8 hours of sleep every night. I don’t regret it.

For me one of the most challenging parts of committing to a bedtime is how often I would crawl under the covers before I felt tired. Many nights, I stared at the ceiling for an indeterminate amount of time before I would fall asleep. This would make it nearly impossible to say with any certainty whether I was meeting my sleep goal. I wanted a way to measure my sleep but at that time, sleep trackers were rare and cumbersome. A few years later, I soldered together some hardware and started trying to track my heart rate throughout the day and night. It was a fun project, but I quickly realized that wearing a heart rate strap 24/7 was impractical and uncomfortable. Additionally, I was going through CR2025 batteries at an alarming rate. I gave up on the project and lost interested in tracking my sleep for a while.

Then in 2016, Mailchimp gave each of its employees a Fitbit Charge HR as a holiday gift. It took me a few months of wear to realize that I now had the sleep, heart rate and movement tracking that I had always wanted. Unfortunately, the FitBit literally fell apart before I had the time to collect any significant data on my sleep habits. I even replaced that FitBit with a second one that also quickly fell apart. I’m told that the higher end FitBit devices are more rugged but I gave up and switched to a Garmin device. The device itself is great. I’ve been wearing it almost continuously for over 2.5 years and rarely experienced problems. I also find Garmin Connect, the web UI where the device data is surfaced, to be a pleasant product for regular use. The one big downside, though, Garmin Connect doesn’t really offer an API. “Enterprise partners” can request access to their Health API. Based on the request form, I assumed that I would not qualify as an enterprise partner. In order to extract the data for my own personal analysis, I had to resort to a bit of reverse engineering. Ultimately, I was able to “liberate” almost two years of continuous sleep, heart rate and movement data. In this post, I'm going to use that data to determine if I've really been meeting my sleep goals in 2019 and to look for ways to improve in 2020.

Sleeping through 2019

This year has been a non-typical year for me and my family. For almost half of the year, we were living in a small, temporary rental while our hurricane damaged house was repaired. I expected to see periods where I struggled to maintain my sleep schedule. During the first half of May, for instance, we had moved out of the rental but were still unable to move back into our house. On the whole though, I expected to find fairly consistent sleep patterns that regularly fell just short of my 8 hour nightly target. I was right.

My sleep times for 2019 are shown below. Days that are missing were not recorded. That generally means that I wasn’t wearing my Garmin watch as I slept.

Deep Sleep

Light Sleep

Awake

Deep Sleep Light Sleep Awake

An Average Night of Sleep

Recall that my goal is to get at least 8 hours of sleep every every night. On average, did I accomplish that goal? Almost. Below are averages for my sleep. Since my sleep patterns are different on the weekends, where I tend to stay up a bit longer and sleep a little later, I've also broken out those averages. The time ranges that are noted are standard deviation, so roughly 70% of nights fall into the range shown.

All Days

AVG TIME ASLEEP 7 hrs 55 mins ± 47 mins

AVG LIGHT SLEEP 4 hrs 49 mins ± 55 mins

AVG DEEP SLEEP 3 hrs 6 mins ± 45 mins

AVG TIME AWAKE 8 mins ± 5 mins

AVG BEDTIME 10:18 PM± 44 mins

AVG WAKE UP 6:28 AM ± 51 mins

Weekdays

AVG TIME ASLEEP 7 hrs 50 mins ± 40 mins

AVG LIGHT SLEEP 4 hrs 44 mins ± 52 mins

AVG DEEP SLEEP 3 hrs 5 mins ± 45 mins

AVG TIME AWAKE 8 mins ± 5 mins

AVG BEDTIME 10:08 PM ± 35 mins

AVG WAKE UP 6:12 AM ± 38 mins

Weekends

AVG TIME ASLEEP 8 hrs 9 mins ± 58 mins

AVG LIGHT SLEEP 5 hrs 0 mins ± 58 mins

AVG DEEP SLEEP 3 hrs 9 mins ± 46 mins

AVG TIME AWAKE 8 mins ± 6 mins

AVG BEDTIME 10:44 PM ± 53 mins

AVG WAKE UP 7:08 AM ± 59 mins

While I'm not quite meeting my goal of 8 hours each night, there is still plenty of reason to be proud of these numbers. In 2017, FitBit looked at similar averages for their millions of users and it seems I'm getting way more sleep that the average FitBit user, which is only 6 hours and 38 minutes nightly.

How to get more sleep?

Looking at the average time I sleep every night, I'm only about 5 minutes off of my goal. If, however, you look at the percentage of those nights when I meet my 8 hour sleep goal, it's only 42.6%. Of the 331 days where I recorded data, 190 of those fell below 8 hours of sleep. So what can I do about those 190 nights? I could go to bed earlier, I could sleep later, or I could somehow reduce the amount of time I'm awake during the night. Unsurprisingly, the biggest opportunity to improve seems to be getting to sleep by 10 pm every night. In fact, if I would have been diligent about being asleep at 10 pm every night, I would have met my 8 hour sleep goal 61.6% of the time. Focusing on when I get up, or trying not to wake up during the night would not have near the same impact. So clearly, in 2020, I need to get my ass to bed. Speaking of which, it's currently midnight and I'm still writing this post. Good night.

Puzzle Corner

2019-10-05T00:00:00Z

The only paper periodical that is still delivered to my house is MIT’s Technology Review. In the back of every issue, there is a section called Puzzle Corner. As the name implies, it includes puzzles. It’s really an amazing column and has a long history; it has been edited by Allen Gottlieb for more than 50 years. The column consists of densely packed reader-submitted math puzzles and solutions to puzzles published in earlier editions. There are typically three puzzles and the first one almost always focuses on board games (i.e. chess or bridge). While I rarely have the time to solve the puzzles, I seldom miss the chance to read through them.

Earlier this year, the May-April Edition arrived and I immediately flipped to the back to skim through the puzzles. My thirteen year old daughter, Zoe, was sitting beside me on the couch and noticed the chess board graphic in the first problem. She was curious why I had skipped all the content to look at this section. I explained how the column publishes puzzles and how I always read the puzzles but rarely take the time to solve many of them. I told her that many of them required knowledge of either chess or bridge and that I had little interest in chess and had never played bridge. “Oh, here’s one you can solve,” I added as I skimmed the second problem.

M/A 2. Nob Yashigahara wants you to place the digits 1 through 9 once each into the nine boxes to yield a valid equation.

Solve

Zoe gave me a confused look, “I don’t know how to solve that.” “Sure you do, you could very easily just use your computer to try every option,” I responded. She looked confused, “How?”

And with that, it has become a bit of a pet problem that we’ve continued to discuss over the next few months as I tried to explain to her why I was so certain that this problem could be solved in very little time using a computer. What follows is a bit of a summary of the explanation and the solution that we found.

How?

Working on math puzzles with a middle school student has it's challenges. Having yet to cover probability in school, the concept of a permutation which would easily tell us how many unique ways exist for sorting a set of 9 numbers is not something we can rely on. We have to start more basic.

We’re going to have the computer make guesses for us until it finds a guess that works. For example, we’ll try

to see if that adds up to 1 and we’ll also try

We’ll try all possible ways to arrange those 9 numbers. How many guesses will we need to make?

Let’s change the problem around slightly to make it easier to figure this out. Let’s start with a list of those nine numbers. [1, 2, 3, 4, 5, 6, 7, 8, 9]. Let’s imagine that the first number in the list corresponds to numerator in the first expression, the second number in the list corresponds to the first digit in the denominator, the third item to the second digit in the denominator and so forth. So, for example, [1, 4, 8, 2, 7, 9, 3, 6, 5] would correspond to,

If we want to try all possible guesses, all we have to do is find all the different ways possible to reorder that list of nine numbers. But how does one find all the ways to order a list of nine numbers?

How many ways can we order 9 items?

Let’s start with a smaller problem, just so we can see the pattern more easily. Instead of finding all the ways to order 9 items, let’s first find all the ways to order just 3 items. When we understand that, it should make it much easier to solve the bigger problem with all 9 items.

So we have the list of 3 numbers, [1, 2, 3]. If we were going to take out a piece of paper and just start writing down all the ways to order them, we need to first choose a number for the first item in the list. Why not start with 1?

[1, ?, ?]

We’ll eventually do the same thing with the first item being 2 and then 3. But for now, let’s count the number of orderings there are when the first item is 1. Since 1 is accounted for, we now have a list of 2 items, [2, 3]. There are only two ways to order [2, 3]. There is [2, 3] and [3, 2]. So our paper list might start like this,

[1, 2, 3]

[1, 3, 2]

And then we need to find all the ways to order the list when 2 is the first number.

[2, 1, 3]

[2, 3, 1]

And finally, we need to find all the ways to order the list when 3 is the first number.

[3, 1, 2]

[3, 2, 1]

So putting them all together, there are 6 different ways to order the numbers [1, 2, 3].

[1, 2, 3]

[1, 3, 2]

[2, 1, 3]

[2, 3, 1]

[3, 1, 2]

[3, 2, 1]

Notice that when we chose the first item in the list, we had 3 options to choose from [1, 2, 3]. And once we had selected something for the first item, we had 2 things to choose from and there are only 2 ways to order 2 items. So the total number of ways to order a list of 3 items is 3 × 2 = 6. Now, let’s repeat the same thing but let’s find all the ways to order 4 items.

Notice that we have 4 items when we select the first item in the list, then we have 3 items when we select the 2nd item in the list and finally we we have 2 items to choose from. So the number of ways to order 4 items is 4 × 3 × 2 = 24

Let’s just follow the pattern. The number of ways to order the 9 items [1, 2, 3, 4, 5, 6, 7, 8, 9] can be found with,

9 × 8 × 7 × 6 × 5 × 4 × 3 × 2 = 362,880

That seems like a lot but computers are very fast and can make that many guesses in less than a second. We’re also still trying more guesses than we really need to, but this is good enough to solve the problem from the Puzzle Corner in a very short amount of time. Using a simple program that tries all 362,880 guesses and prints out solutions to the problem finds 6 solutions!


    5/34 + 7/68 + 9/12 = 1
    5/34 + 9/12 + 7/68 = 1
    7/68 + 5/34 + 9/12 = 1
    7/68 + 9/12 + 5/34 = 1
    9/12 + 5/34 + 7/68 = 1
    9/12 + 7/68 + 5/34 = 1

It's probably pretty clear by looking at the 6 solutions why our simple program is still trying too many guesses. Addition is communicative, so for example,

Removing the redundant guesses

What follows is not really necessary as we’re obviously able to compute the solution to the problem in a very short amount of time. It’s pretty satisfying, though, to make our program a little bit more efficient and also have it print only one definitive solution to the problem.

The issue is that we found all the unique ways to order the list of 9 numbers, but some of those orderings are actually redundant because what we really want to find is all the unique ways to order those 9 numbers into 3 sets of 3 numbers and the order of those 3 sets does NOT matter. There are a number of ways to eliminate the redundancy, but one way that is pretty easy to integrate into our existing solution is to restrict the order. Let’s only try orderings if the sets of 3 numbers are sorted by their minimum value. So if you consider the example given above,

[5, 3, 4, 7, 6, 8, 9, 1, 2]

min [5, 3, 4] = 3

min [7, 6, 8] = 6

min [9, 1, 2] = 1

The minimum value in the first set of 3 is 3, the minimum value in the second set is 6 and the minimum value in the last set is 1. This ordering is not sorted properly since the minimum value in the last set of 3 numbers is smaller than the minimum value in the first set of 3 numbers. This should not be one of our guesses. Consider, however,

[9, 1, 2, 5, 3, 4, 7, 6, 8]

min [9, 1, 2] = 1

min [5, 3, 4] = 3

min [7, 6, 8] = 6

This would be considered a valid guess since the minimum of each set of 3 is in sorted order, 1 < 3 < 6. Integrating that into our little program and running it again, this time we get a single solution.


    9/12 + 5/34 + 7/68 = 1

So how many guesses did we have to make now that we’ve eliminated the redundant guesses? A good clue is that previously we got the same answer 6 times and now we only get a single answer. If your guess is 362,880 / 6 = 60,489, then you are correct. The reason is that we have 3 sets where the order doesn’t matter. You can arrange 3 sets in 3 × 2 = 6 different ways. If we then removed all of those orderings, then we reduce the number of guesses by a factor of 6.

Additional Notes

Alan Gottlieb, editor of Puzzle Corner and professor of computer science at NYU, maintains a page about his column that includes the history, ground rules and all the back issues.

When we were trying to figure out how many ways one can re-order a set of 9 numbers, we had to restrict ourselves to middle school math. However, this act of arranging a set of items into an order is known as a permutation. For a set of N items, there are N factorial possible permutations.

PHP and the case of swallowed errors

2018-08-20T00:00:00Z

In an effort to write more, I'm considering cataloging more of the things I learn on a day-to-day basis. This post certainly falls into that category of posts. We recently solved the mystery behind a pattern of failures that have been happening very infrequently over a long period of time. The culprit was surprising and a little unsettling, so I thought it might be worthy of a public blog post. Before I dive into the details, though, let me be very clear that while I contributed to this investigation this work is not mine alone. As we accumulated findings each time these issues would surface, many folks in our engineering group ended up adding to the pile of insights that would eventually lead us to a bug PHP's mysqlnd database driver. At the time of writing, the bug has been reported a number of times but still exists in the latest version of PHP (7.2.8).

“something weird happened during failover.”

Like many shops that use MySQL we run pairs of servers in a master/master configuration. At any given time, we are only writing to one server (the primary) and if we happen to lose that primary, we failover to the secondary and continue running. We try to minimize the impact of these failovers but they do not happen transparently. For most long running jobs, a failover means that the job receives an error, it then terminates and can be later retried, operating on the secondary database that has taken over for the dead primary. A few months ago, we had a database primary go down. During the cleanup that followed, we found the data associated with a particular job that had to retry was in a completely unexpected state. Here's a rough sketch, in pseudocode, of what that job looked like.

While the real code is obviously much more complex, the basic flow here is simple. In the database is a number of items belonging to the job. The job does a query that selects all the items that have not been processed. It then processes each of those items, marking each one completed. Once it completes all the items, the job is marked as completed and it will not run again. If we had a failover in the middle of running this code, we expected that the job would fail either through an exception or a fatal error. The job would then be restarted and it would pick up processing where it got previously interrupted and only after it successfully reached the end of the query results would it mark the job as completed.

Instead we found jobs that were marked completed that had some number of items that were not processed at all. Since the job was marked completed, it was clear that the code above had successfully iterated through the entire loop. This caused us to initially suspect this was an issue with asynchronous replication. But in some cases, we even saw that none of the items were processed. That would have indicated significant lag in our replication that just wasn't realistic. The cause would remain a mystery for a while longer since the impacts were still small and we had no reason to believe the issue wasn't isolated to this particular job.

“that looks familiar”

A few months later, though, an attempt to lower some timeouts on database connections would clarify what was causing the grief. Shortly after pushing out configuration changes to lower the timeouts, we observed a very similar pattern happening in another place in the codebase. It appeared that if the connection timed out in the middle of a result set, we were not getting a proper error so a variety of problems surfaced all from having erroneously assumed that the entire result set had been returned. Much of this code was also well over a decade old and had not changed much at all since it was written to run on PHP 5.3. It was certainly still possible that a recent change had caused all this grief but all the signs pointed to PHP behaving in a way we had not anticipated. While I fully expected to simply rule out PHP's error handling as a culprit, I set about answering the question: How does PHP handle MySQL completely going away in the middle of returning a query?

Reproducing the issue

I created a controlled environment for the experiments. This included a PHP node for running queries, a MySQL server node and a proxy server written in Go that made it easy to programmatically disconnect MySQL to simulate failure. In each of the tests, I would execute a straight-forward SELECT statement that was expected to return 10,000 rows. The proxy would allow a fixed number of bytes to be returned to the PHP client before closing the socket in both directions. The PHP code in question looks something like this.

Given that we use PDO::ERRMODE_EXCEPTION, we would certainly expect that an exception will be thrown the moment the proxy closes the connection. We would not expect to see this example print the output “Number of rows returned". And we would certainly not expect to see it print output with a number less than 10,000. But when you run this example, you see the following output:

Number of rows returned 0

🤦‍

That didn't go well but why zero and not some arbitrary number?

Buffered vs Unbuffered Queries

PHP's MySQL drivers support two query modes, buffered and unbuffered. In buffered mode, calling execute causes all the results to be transferred back to PHP and stored in memory before the call to execute will return. This is the default mode and is generally favored in that it allows the result set to be treated like a plain ole iterator in PHP (including rewinding and counting rows). As you might expect, buffered mode is problematic if the result set is large. In unbuffered mode, execute returns before the results are transferred back to PHP and the results are read in from the socket as the code iterates. The biggest downside of an unbuffered query is that the connection cannot run any queries while iterating over a result set.

Returning to the example above, the reason why we got zero rows is that we were running in buffered mode. The disturbing part here is that no exception was thrown and no error occurred even though we got none of the rows that we should have gotten. So what happens if we change into unbuffered mode?

Number of rows returned 98

😡

This time we received 98 of our expected 10,000 rows and, again, no error or exception. This sucks! This means that we can successfully iterate through a result set in PHP and still not know if we have seen all the rows that were returned. It should also be clear at this point that the original job that tipped us off to this problem was using unbuffered queries.

Note that the code for the controlled environment that reproduces this error is in Github.

“It's a just a damn warning!”

One thing that is hard to miss about running the examples above is that they produce a warning in PHP with the text “Empty row packet body”. Clearly PHP is aware of the error condition, so I hoped that we were just missing some configuration that prevented it from being promoted to an exception. But alas, no such luck. You can tell by these lines of C that the MySQL driver only ever emits a warning when there is an erroneous socket read. That's not good.

Applying duct tape

After some research, I found this was definitely not an unknown PHP bug. I'm a little surprised this behavior is not more widely known. I guess it really reinforces that failures are relatively rare and receive very little testing. I found three different reports in the PHP Bug Tracking System about this issue, #64763, #66370 and #65825. In one case there had been an attempt at a fix, but the patch was later abandoned. You'll also notice the bug is present in PHP 7.2.8 which is the latest release as of the time of this writing. However, I was able to reproduce the bug on every version of PHP that used the mysqlnd driver. I suspect this case worked properly when PHP used libmysql, but I have not been able to confirm that.

We ultimately decided that this behavior was just too dangerous for us. We have a PHP codebase that has been around for over a decade and reviewing all of our result set handling is no small task. Instead, we are using a custom error handler to promote the warning to a PDOException. This is a solution we have all agreed is “gross but necessary”. Hopefully, we'll see a proper fix in a future PHP release and we'll be able to remove the duct tape, but for now it looks like our best course of action.

S(h)ave The Beard?

2017-01-29T00:00:00Z

I read on the Internet that the beard trend is over. That's great news because I'm not trendy and I decided to stop shaving at the beginning of the year. I made it a few weeks without arousing any suspicion that I intended to keep the beard, but now it's time time decide: do I s(h)ave the beard?

First I asked my family. The kids were overwhelmingly in favor of keeping the beard. My wife, on the other hand, wasn't so easily convinced. She reminded me that there are brand new razors under my sink and asked if I needed any help finding them. So the vote among family members was 4 to 1, but Stephanie claimed that her vote counted for more than everyone else's, so let's call it 4 to 4. Surely I'll have better luck if I extend voting to the Internet. I mean democracy hasn't failed us lately, right?

So help me break the voting deadlock in my house. Should I keep the beard or not?

BTW, cute cartoons of me are the amazing work of bitmoji.

Also BTW, I'm not going to do anything with your email address except send you a link to vote.

The Holy Fear

2015-12-03T00:00:00Z

As texts for sharing useful guidance and wisdom, religious texts are pretty much the worst. Visit the front page of any news site right now and there is a decent chance that much of the reported violence is rationalized by the teachings of one of these texts. Meanwhile, my Facebook feed is an absurd mix of age-old intolerance and cherry-picked biblical texts about peace and understanding. It saddens me that much of the world still pays deference to the non-sense writings of lunatics, but I have to admit the absurdity is absolutely fascinating. A few weeks ago I was telling a friend of mine that an algorithmically generated Bible of nonsense would probably still produce the same base beliefs as it really needed to be abstract and meandering enough to rationalize any action. The idea stuck with me and I eventually concluded that there was a need for a new religious text that explicitly addressed more modern vices, like out of control drug use and hotel room destruction.

So I went ahead and remixed my own religious text using what I might call Markov Chain guided Mad Libbing with two source texts: The King James version of the Holy Bible and Hunter S. Thompson’s Fear and Loathing in Las Vegas. Unsurprising, the resulting text is just as arbitrary and probably a bit more entertaining. Consider:

“Happy shall he be, that taketh and dasheth thy little ones against the stones.” Psalms 137:9

Oh sorry, that’s from the original King James Bible. That’s real guidance and wisdom. My religious text, The Holy Fear, is almost 900 pages of pure wisdom and debauchery woven together with such bullshit that it rewards the most imaginative interpreters.

“But it shall be, when l cometh on, he shall wash himself with rage: and when the maid is very, he shall come into the rear again.” Riderunners 23:11

As usual, the programming associated with this blogpost is in my github. In keeping with religious tradition, I have not read my own religious text completely so if you find inspiring passages, please do point them out to me on Twitter.

credit: bible text by Alexander Smolianitski

The Holy Fear

The sacred union of the King James Version of The Holy Bible and Hunter S. Thompson's Fear and Loathing in Las Vegas

Generated from Magic Seed

0x420

Don't Lose $2,000 to Comcast Business

2015-01-09T00:00:00Z

This is a post I don't even want to write. I like building things and writing about building things and I generally don't like bellowing to the Internet when something doesn't go quite as well as expected. But the more I've chatted with people about my experience with Comcast Business, the more I realized that it's probably worth writing a bit just to let others know what to expect. So consider this a cautionary tale that gives you some idea of what to look for if you have to deal with Comcast Business.

Some Context

To give you the brief backstory, I became a Comcast Business customer in May of last year. I had been happily using Comcast consumer internet for a while but there were a couple of things that convinced me that it was time to make the move to business class. The first was the fact that I work remotely and my Internet service is extremely important to me. It is also the way that I communicate with the rest of my team, which often happens over video conference. There were days when I would spend around six hours connected via video conference to our Brooklyn office. The other thing that convinced me was the fact that Comcast had recently introduced data caps in the Atlanta area limiting customers to just 300G of transfer per month. Despite how I or anyone else feels about these caps, I knew they were put in place because of people like me who were using more bandwidth than others. However, it's also important to note that I never actually exceeded the 300G in a month. I only came close. I could have stayed on consumer Internet and, in hindsight, I really should have stayed on consumer Internet as it would have saved me a lot of grief and an awful lot of money.

Tip #1: Beware, There is a Contract

When I first contacted Comcast Business about getting a business account, I was put in touch with a salesman (Eric) over email who explained the options to me. The price of the plan is based on speed and whether you are also interested in receiving business phone service. Eric sent me the following graphic and indicated that I just need to select which package best suited me.

You will notice that the two columns are not labeled. This is how Eric sent them to me and I later found out that the left column is Internet only, the right column also includes phone service. Let me point out a couple of things about this graphic. First, it is copied directly from Comcast Business Plan & Pricing Page. More important is the fact that these prices are predicated on a minimum 2 year contract. That fact is not included anywhere in that graphic and that's usually a thing you don't just exclude from your advertising. I have no idea if it is a thing that can be legally exclude from advertising, but just be aware that the minimum contract term is 24 months. I mention this specifically because this is the first place they nailed me. Eric offered to send me an “order” to schedule the install. What I did not realize until later is Eric sent me a 36 Month contract. Shady on their part, but I really should have scrutinized the document he sent me to sign because it definitely says “36 months.” So first important tip, make sure you get the minimum of 24 months if you have to deal with these guys.

Tip #2: You pay even if they don't give you service

This was the big one for me. So I'm on the hook for a 36 month contract. This is fine by me. Like I said, I work remotely and my Internet service is really important to me. If they continue to give me good service, I'm fine with continuing to pay them. But in October I moved out of Atlanta. In fact, I moved out of Georgia and hopped two states over to North Carolina. During the move, I called Comcast Business to have my service moved. This was when things got interesting. The customer service representative informed me that the area I had moved into was “Charter territory” and that they were unable to move my service. I told her I was kind of bummed because the service had been great in Atlanta. I asked her what I needed to do and her response left me stunned. She explained that even though they were no longer able to uphold their part of the contract, I would still needed to pay them. The best they could offer me was an early termination fee which was right around $2,000! She must have gotten really annoyed with me. I continued to ask the same question in slightly different terms:

“$2,000, for you not to give me service!?”

“So you cannot uphold you part of the contract, and you want me to pay $2,000!?”

The answer, as you might expect, is "Yes". The customer representative claimed that relocation to an unserviced area is explicitly spelled out in the terms of service. It's not. I can't blame her, though, I would have said anything to get off the phone with me.

So I'm out $2,000, you should beware

So Comcast Business got me. They are $2,000 richer and I'm frustrated and unhappy. I can take the blame on being fooled by the salesman shenanigans but I had no way of knowing that Comcast would force me to pay even if they failed to deliver what they offered. Like I said, I work remotely so I cannot simply go without Internet. I had to setup service through Charter even while I continue to pay Comcast. Many others out there will be forced to move to Comcast Business. They're a virtual monopoly so I cannot realistically advise that you avoid them at all costs. They will continue to use data caps to force those of us who use more bandwidth to move to business accounts. If you get forced, I have two bits of advice for you.

Make sure you sign the minimum 24 month contract.
Try to find someone willing to take over your contract should you find yourself needing to move to a non-serviceable area. The customer representative did suggest that they are transferable. I have no idea if this is true. But most of all, keep in mind that you may be paying out your contract even in situations where Comcast is not giving you service.

For the moment, I continue to pay Comcast monthly for service they will not give me. I am trying to decide the right approach to take. Should I just pay the $2,000 and be done with this nightmare? Or should I find someone who could use good Internet, give them the $2,000 and let someone benefit from my troubles and money? I don't know. But if you have to do business with them, do be very careful.

SURFing The Charles

2014-11-30T00:00:00Z

Earlier this month, I assembled some photos of the Charles River into a time lapse montage. It was a fun project even if it wasn't very novel. A while back a friend of mine suggested that I write more about the construction of these little projects. I figured I would give that a try with this last project. But here's the thing: I'm not really an expert at any of this stuff I tinker with. In fact, I'm not really an expert at anything. But I will try. Computer vision experts, please forgive me.

So as I said in that initial post, I recently discovered a whole DVD full of images of the Charles River taken over the course of a single day. When I found them I imagined taking a particular sequence of images where there is a great deal of change (i.e. sunrise) and assembling one composite image from vertical strips taken from each of the images. This has the benefit of, hopefully, keeping the scene recognizable but also orienting time from left-to-right, which is certainly the familiar orientation for English speakers. (I am told there is no intuitive natural mapping for right-to-left languages, but this is beyond both my knowledge and the scope of this post.) This would obviously be a pretty trivial task if all of those images had been taken from a stable vantage point but that wasn't really the case. The camera was, in fact, on a tripod, but the photographers were eyeballing the angle every few minutes when they took a series of photos from different angles. I hand selected about 80 morning images from roughly the same angle looking east toward Back Bay. While they were “roughly” the same angle, there was still a great bit of variation as you see in the map below of their final computed translations. The effect of the tripod is pretty apparent as their is much less vertical variation.

Finding those transformations

The transforms you see above represent the trickier part of the final product. We start with just a bunch of photos of equivalent size and we have to use the contents of those photos to figure out where the camera was pointing for each. Well, ok, to be honest, we just need to know how the camera changed from each photo to the next. If we can figure that out, we can simply move (transform) each image as if they were all taken from a stable position. This is not a unique problem. If you have ever seen those fancy programs that will take a bunch of photos and make a panorama for you, this is essentially the same problem (see Image Stitching). There is some interesting theory behind all of this, but as I said, I'm no expert in computer vision. Instead, I'm going to keep this practical and avoid much discussion of the theory. Instead, we'll just go through this step-by-step.

1. Finding features

If you were to align all of these photos by hand, you might focus on particular features in the image and just move the images until those prominent features align properly. A good feature to look at is the Prudential Tower. If I were wise, I would have simply hand aligned all of these images using that one building as the guide.But that's not really any fun. I was also trying a lot of different things as I tend to do with these projects so I would have lost the ability to experiment with other views quickly.

I could have aligned each pair of images by hand, using the Prudential Tower.

Computers, as you are probably aware, have a harder time recognizing buildings. But there are other techniques for finding memorable features in images. In fact, there are enough techniques to warrant a table in Wikipedia. Beyond just finding interesting points in an image, though, we also need to be able to describe that interesting point with enough detail that we can find it in a second image. Seeing where a feature moves from one image to the next is the whole basis for this stuff. This interesting point with an accompanying description is commonly called a feature descriptor. This consists of a point of interest and a “feature vector” which describes the feature in a way that we can compare it to other features to determine if it looks similar. There are really two such feature descriptors that are pretty popular today and easily accessible: Scale Invariant Feature Transform (SIFT) and Speeded Up Robust Features (SURF) . These two are quite similar in nature and both have the property that the feature descriptors are immune to some transformation. In other words, they still match pretty well when you move the camera a good bit. I used SURF for this project, but I could have just as easily used SIFT (or another descriptor). There are some tradeoffs, but they are certainly beyond the scope of this post. To give you a sense of what a computer finds interesting in a photo, here are the features from one of the photos in the sunrise sequence.

SURF features in the 8:37AM photo slice

If you are using OpenCV, all of this is built in and you can use code similar to what is shown below. If you want a guide that attempts to bridge the gap between theory and code, I recommend Programming Computer Vision with Python.

2. Matching features

Moving onto the next step in this adventure. We have a set of descriptors for each image. Now it's time to find those same descriptors in other images. In my case, I'm most concerned that adjacent images align properly. For other problems, it is common to match all other images to a single reference image. But I need only match the features from one image to the one that will appear to its right. The nature of these feature descriptors (aka feature vectors) is that it is pretty easy to determine the similarity of any two. So a slow way of matching features is to take each feature in one image and look for the most similar feature in the other image. If those two features are within a distance threshold, then you have yourself a match. There are, however, much faster ways of doing this that rely on well used indexing techniques. One method that works pretty well is FLANN (Fast Approximate Nearest Neighbor Search). Like SIFT and SURF, this also has the advantage of readily available in OpenCV. Using this method, we'll get back a list of matches containing pairs of features, one from the first image and the other from the second. So for each feature, we now know how far and in what direction it moved. Again, using OpenCV, the code is straight foward.

And here are some of the matches for a pair of images in the sunrise sequence.

Matches found between the 8:01AM and 8:21AM slices

You will notice that many of the matches are absurdly wrong. A great many more, though, are correct. When we look for a proper transform next, we'll have to take care to use strategies that minimize the effects of these outliers.

3. Finding a transform

Generally when you arrive at this step, we assume that the camera will have moved with absolute freedom. The photographer may have rotated, or moved to the side, or taken a few steps back. So in general we are looking for a transformation matrix that will correct for all of these camera movements. In fancy terms, we say these two images are related by a homography which we can find with some math (generally using RANSAC) and then use the resulting matrix to transform one image into the coordinate space of the other image. That's fancy and perfectly tractable, but I was looking for something a bit simpler. First, I could tell that most of my images were primarily affected by translation. There wasn't a lot of rotation or scale change, they had simply moved left or right. There is a bit of rotation, of course, because this camera was atop a tripod. But accuracy was not the primary goal of this project. I did not want to deform the images with 3D transforms. I really wanted the strips to be more of a gallery for the original photos which makes my job a lot easier. I simply need to find the translation of each image relative to the image to its left. I could have certainly gone ahead and computed the homography and then extracted the translation from there, but why bother? I do something far sillier; I compute the median translation between each image. The fact that gross outliers exist, by the way, is why I prefer median over mean. So now I have for each image the distance it needs to be moved in both the x and y directions. This is what you see in the figure with all the rectangles shown above, each image's bounding box relative to the other images. So now the tricky part is done, nothing to do but paint some pixels.

Wrapping it all up

With these projects, there is always a long tail of finishing touches. I obviously built a little web app for displaying the images and their associated time. I also had a few other touches to deal with. The first is that it can be tricky to find strips on the right side of the image. If images later in the sequence are translated too far to the left, we'll end up with an empty strip. To fix this, I did the dumbest thing that worked. If an image is too far to the left to produce a good vertical strip, I discard it and rebuild the strips as if it wasn't there. Keep in mind, I don't have to find the transforms for the new images since I already have global transforms for each image. The next issue is a purely aesthetic one. There is the issue of that one image where the sun makes its debut over Back Bay. As you might expect, it didn't just work out naturally so that strip aligns just right to show off the sun in the composite image. Nope, I cheated like hell by translating the whole image to the left until I had the sun showing. I even had to discard some images from the right to make sure it all worked out. That translation was a pretty manual process with several code tweaks, so I did not include it in the code I put up on Github.

The Source Code

As with most of my little projects, I've released the associated code on Github. With the exception of the feature descriptors, most of this code is somewhat straightforward to write by hand (it's a little trickier if you want it to be super efficient). Fortunately, though, OpenCV has an excellent collection of everything I've described. It even has bindings for Python if you aren't into C++. For this project, I had originally written some parts by hand before realizing I didn't have to and moving everything to OpenCV. If you are not on OSX, I have to apologize as the graphics code uses CoreGraphics which will prevent it from compiling on anything but a Mac. It would be fairly simple to do the same thing using Cairo or Skia.

And that's about it. A DVD full of images, some hand curation, some OpenCV code and I was able to stitch them together into a little time-lapse panorama without knowing much at all about computer vision.

A Morning On The Charles

2014-11-08T00:00:00Z

In 2006, my friend Myoung Lah and a group of housemates set a tripod atop Baker House at MIT and painstakingly moved the camera by hand at regular intervals to produce about 1,000 different photos from 6 distinct angles along the Charles River. They apparently did this in shifts for almost 24 hours. I'm still not really sure why they did it or why they didn't automate it to ensure consistency in the shots. By this time, though, I had graduated and moved back to Atlanta so I did not have the opportunity to ask Myoung: “why did you do this and why didn't you automate it to ensure consistency in the shots?” It wasn't until Myoung reached out to me asking for some advice on a web app he was building to display the images that I even knew they existed. I happily helped him with the problem he was having and, in exchange, I asked if he would send me copies of the photos. A few weeks later, a DVD arrived in the mail and I discovered some truly incredible shots of the Charles River. I remember wanting to do something with the photos back then, but I stuffed the DVD into a drawer and promptly forgot it existed.

But last month, almost 8 years later, I began preparing for a move to North Carolina and found the forgotten DVD tucked away in a stack of other discs. I immediately put it in my DVD drive to make sure it was still in good shape and promptly copied the photos onto a shared drive so I could easily look through them. I had failed to do anything constructive with the images the first time around, so I figured I would try to assemble them into something remotely interesting before I had a chance to forget them again. There was a particular sequence from about 5:30am until after 8am when the sun can be seen rising over Back Bay. In one these photos, you see the sun make its first appearance by sneaking in between two buildings in the skyline. The only issue was that despite their best attempts and despite having a tripod, the camera angle varied quite a bit even over the few hours of the sunrise sequence. I wanted to take a slice from each photo and build a composite image with the time progressing from left to right so that on the left you could see the darkness of pre-dawn progressing through the bright oranges of sunrise and finally into the blue skies of daytime. I hoped with enough tweaking, I could even feature the sun peaking through the buildings in one of the slices. But first I needed to deal with the photo angle issue. I optimistically tried some naive approaches, then considered just manually aligning the images, but finally I took an algorithmic approach. I will save the gory details of that for another post, though. I will note, however, that my goal here was not produce a seamlessly stitched image. In fact, I try to avoid any transforms that would distort the original image. I was thinking of this more as a way to organize the original photo sequence than a way to magically smoosh them into one. To define this more technically, my image registration is based on just a translation matrix and not a general homography matrix.

So what you see above is a sequence of images taken over a three hour window on the morning of November 3, 2006. Clicking on any of the slices will show you the original photo still aligned with the other images. If you like, you can also move between original images. In my time at MIT, I probably saw far too many of these sunrises. This would be about the time I left the lab many mornings. Thanks to the effects of sleeplessness on memory, though, I remember fewer of them. It's nice to have one so vividly preserved. And thanks to Myoung and friends for moving that camera (even if we still have no explanation for why they did it).

Thanks to Stumm for reviewing early versions.

The Pleasant Places to Live

2014-02-03T00:00:00Z

Where in the U.S. will you find the most “pleasant” days in a year?

“pleasant” here means the mean temperature was between (55° F and 75° F), the minimum temperature was above 45° F, the maximum temperature was below 85° F and there was no significant precipitation or snow depth.

5 Most Pleasant Places

LOS ANGELES, CA 183 days/yr
SAN DIEGO, CA 182 days/yr
OXNARD, CA 166 days/yr
SIMI VALLEY, CA 156 days/yr
SAN FRANCISCO, CA 153 days/yr

5 Least Pleasant Places

MC ALLISTER, MT 14 days/yr
NORTHEAST OF RENO, NV 15 days/yr
CLANCY, MT 15 days/yr
DOUGLAS, WY 15 days/yr
EAST OF CEDARVILLE, CA 16 days/yr

It has been a winter of dreadful weather so far. I spent January flying back and forth from New York expecting to find a different set of conditions at the end of each leg. Whichever way I went, bitter cold greeted me at the end of the jet way and often with a coating of slick ice. It’s hard not to dwell on anomalous and unpleasant weather. It got me wondering, though, where in the U.S. do you go if you want the most “pleasant” days in a year?

I decided to take a stab at what constitutes a “pleasant” day and then aggregate NOAA data for the last 23 years to figure out the regions of the United States with the most (and least) pleasant days in a typical year. The results, I think, are not that surprising and pretty much affirm the answer given off the cuff by many of my west coast friends when asked about the best places, "Southern California?" For the areas with the least pleasant days, I admit I would have guessed North Dakota. However, it's much of Montana that gets an average of a couple of weeks of pleasantness each year. I'm sure, though, they would shake their frost-bitten fingers at me and remind me that not everyone can take the overwhelming heat of 55˚ F. True, there is a bit of subjectivity to the range I selected. It's just that a lot of subjects share that same preference.

Data: NOAA Global Summary of the Day data archive

Puzzling

2013-11-11T00:00:00Z

A few weeks back, I took some time off for a bit of recharging. That gave me some time to do things that might normally get prioritized out of my schedule, like familiarizing myself with other growing startups in the Atlanta area. I set out reading a number of startup blogs when I stumbled across one with a coding puzzle. I read the description and actually closed the tab quickly. I am not a huge fan of puzzles but when in the right mood (and when not constrained by time), I'm susceptible to the DOS attack they can impose on the analytical mind. I almost escaped too, but I had read just enough to think to myself, “oh, that looks like a constraint propagation problem.” That’s when I gave in and just returned to the puzzle; I knew I had been hooked.

To be honest, I'm not particularly good at puzzles but after conducting hundreds of technical interviews in my career, I've become fascinated with how the mind is able to start at "wut?" but eventually end up at a mental model that leads to a solution. Watching people work through my interview questions, I noticed that the first approach that most people take is usually based on a prior experience with a different, but similar problem. Sadly, I also noticed there is a bias toward more recent prior experiences. If the candidate has insertion sort on the brain because of something they did last week, it can be really hard to NOT see a problem as related to insertion sort. Personally, I think many people can discard those biases if given more time. Sadly, more time is something you are rarely at liberty to give in an interview situation. The fact that I had immediately jumped to constraint propagation was fascinating to me. Why had my mind decided to start there? I ended up writing the code partly to see if my initial approach was even sound but also as a kind of self-revenge for all those poor people I've interviewed. (Just to be clear, though, I never asked puzzlers during interviews.)

The problem

There is a backstory here about a puzzle party, but I’ve skipped all that since, you know, this all happened on my week off. Here is the TL;DR version of the problem instead.

We have ten separate lists of symbols.

[:club, :star, :grapes, :seven, :seven] [:cherry, :crown, :spade, :heart, :star] [:club, :horseshoe, :grapes, :bar, :star] [:bell, :cherry, :club, :seven, :grapes] [:horseshoe, :dollar, :club, :diamond, :bar] [:cherry, :diamond, :cherry, :club, :star] [:bar, :horseshoe, :spade, :grapes, :club] [:crown, :diamond, :cherry, :club, :heart] [:spade, :crown, :club, :grapes, :bell] [:seven, :club, :star, :diamond, :bar]

We need to find a way to map each symbol (13 in all) to a unique character such that each of these lists represents a valid word in the North American Competitive Scrabble Word List.

Initial Thoughts

As I mentioned earlier, I quickly realized this problem could be thought of as a Constraint Satisfaction Search. In fact, the whole thing reminded me of a famous piece written by Peter Norvig several years ago about Solving Every Sudoku Puzzle. What I mean by this is that we will have to cover a search space of considerable size, but there are inherit constraints in the way the problem is structured that will allow us to easily eliminate the vast majority of the possibilities without even checking them. As an example, let us say I want to take an approach where I consider every permutation of 5-letter words in the word list. That’s a lot of checking. I would need to do around a whopping 10^32 checks. That number is way bigger than a bread box. But let’s look closer at how we can cheat. As part of the search we will need to, for instance, consider whether the word “glass” works for the first list of symbols. In order for [ :club, :star, :grapes, :seven, :seven ] to map to “glass”, it must be true that :club is “g”, :star is “l” and so on. Propagating these constraints forward as we search means that our search space collapses very quickly. Just considering the constraints imposed by picking a value (“glass”) for the first list of symbols, the possibilities for the third list of symbols ([ :club, :horseshoe, :grapes, :bar, :star ]) reduces from 8938 possibilities to 2 ("gnarl" and "grail").

Representation

It only took me only a few minutes to determine the basic high level approach that I wanted to take, but thinking about the data structures was a bit more involved. The key question I need answered by my choice of data structure is of the form: If [:club, :star, :grapes, :seven, :seven] is "three" and [:cherry, :crown, :spade, :heart, :star] is "flush", can [:club, :horseshoe, :grapes, :bar, :star] be "funky"?

Thinking about this as assignment makes this easy as we need only a pair of dictionaries to keep track of the symbol <=> character mappings that result from the assignment. The next assignment in the search simply accumulates its associated mappings into the same pair of dictionaries. Detecting invalid assignments is also trivial as we simply watch for conflicts when we merge our dictionaries. And lo and behold, if you cover all the rules without any conflicts we have ourselves a solution.

"three"
{ :club   <=> 't',
  :star   <=> 'h',
  :grapes <=> 'r',
  :seven  <=> 'e' }

"flush"
{ :club   <=> 't',
  :star   <=> 'h',
  :grapes <=> 'r',
  :seven  <=> 'e',
  :cherry <=> 'f',
  :crown  <=> 'l',
  :spade  <=> 'u',
  :heart  <=> 's',
  :star   <=> 'h' }

"funky"
{ :club   <=> 't',
  :star   <=> 'h',
  :grapes <=> 'r',
  :seven  <=> 'e',
  :cherry <=> 'f',
  :crown  <=> 'l',
  :spade  <=> 'u',
  :heart  <=> 's',
  :star   <=> 'h',
  :club   <=> 'f' }

Initial Constraints

Having now formulated this problem as one of constraint satisfaction, it was clear that a number of constraints could be enforced before the searching even begins. This pruning is important as it removes search space that would have to be repeatedly eliminated. That would be wasteful and slow. Before any pruning, there were 8938 possible assignments for each of the 10 lists of symbols but there were two sets of constraints that were pretty straight forward.

The first was simply to eliminate any words that could not be assigned to the list of symbols at all. [ :club, :star, :grapes, :seven, :seven ], for instance, requires a double letter suffix which rules out a lot of assignments. That pass lopped off a significant number of possible assignments.

[ :club, :star, :grapes, :seven, :seven ]

160

[ :cherry, :crown, :spade, :heart, :star ]

5,839

[ :club, :horseshoe, :grapes, :bar, :star ]

5,839

[ :bell, :cherry, :club, :seven, :grapes ]

5,839

[ :horseshoe, :dollar, :club, :diamond, :bar ]

5,839

[ :cherry, :diamond, :cherry, :club, :star ]

209

[ :bar, :horseshoe, :spade, :grapes, :club ]

5,839

[ :crown, :diamond, :cherry, :club, :heart ]

5,839

[ :spade, :crown, :club, :grapes, :bell ]

5,839

[ :seven, :club, :star, :diamond, :bar ]

5,839

For the next set of initial constraints, I considered the character positions where a symbol appears to eliminate some more impossible assignments. Consider that :star appears in the second, third and fifth positions. That means that :star can only be assigned something in the intersection of characters found in the second, third and fifth character positions of all possible words. After filtering out the assignments that missed the boat on this set of constraints, things are looking pretty good for search.

[ :club, :star, :grapes, :seven, :seven ]

134

[ :cherry, :crown, :spade, :heart, :star ]

3,633

[ :club, :horseshoe, :grapes, :bar, :star ]

3,218

[ :bell, :cherry, :club, :seven, :grapes ]

2,708

[ :horseshoe, :dollar, :club, :diamond, :bar ]

4,715

[ :cherry, :diamond, :cherry, :club, :star ]

144

[ :bar, :horseshoe, :spade, :grapes, :club ]

5,083

[ :crown, :diamond, :cherry, :club, :heart ]

4,892

[ :spade, :crown, :club, :grapes, :bell ]

4,631

[ :seven, :club, :star, :diamond, :bar ]

1,633

Putting it all together

The original blog post was written in ruby. Besides a few Rakefiles, I had not written any ruby in about 8 years but I figured it was worth a try. Plus, I discovered that someone went and added eachwithobject to the core libraries since my last ruby adventure. That's pretty helpful.

The result is a bit terse, but not so bad.

And running it gives us…

./solve.rb:129:in `block (2 levels) in <main>': undefined method `[]=' for nil:NilClass (NoMethodError)

Dammit, I mean ... running it gives us...

Solution threficaolusn club => t [:club, :star, :grapes, :seven, :seven] => three star => h [:cherry, :crown, :spade, :heart, :star] => flush grapes => r [:club, :horseshoe, :grapes, :bar, :star] => torch seven => e [:bell, :cherry, :club, :seven, :grapes] => after cherry => f [:horseshoe, :dollar, :club, :diamond, :bar] => ontic diamond => i [:cherry, :diamond, :cherry, :club, :star] => fifth bar => c [:bar, :horseshoe, :spade, :grapes, :club] => court bell => a [:crown, :diamond, :cherry, :club, :heart] => lifts horseshoe => o [:spade, :crown, :club, :grapes, :bell] => ultra crown => l [:seven, :club, :star, :diamond, :bar] => ethic spade => u heart => s dollar => n Solution threficaolusp club => t [:club, :star, :grapes, :seven, :seven] => three star => h [:cherry, :crown, :spade, :heart, :star] => flush grapes => r [:club, :horseshoe, :grapes, :bar, :star] => torch seven => e [:bell, :cherry, :club, :seven, :grapes] => after cherry => f [:horseshoe, :dollar, :club, :diamond, :bar] => optic diamond => i [:cherry, :diamond, :cherry, :club, :star] => fifth bar => c [:bar, :horseshoe, :spade, :grapes, :club] => court bell => a [:crown, :diamond, :cherry, :club, :heart] => lifts horseshoe => o [:spade, :crown, :club, :grapes, :bell] => ultra crown => l [:seven, :club, :star, :diamond, :bar] => ethic spade => u heart => s dollar => p Done ... 1.584249 sec

Clearly, the plan to throw away most of the search space had a significant payoff. In fact, the search is able to cover the whole of the search space by checking only 140,261 possible solutions. That fits pretty nicely in a breadbox. This seems to me a perfectly good way to solve this problem, but again, I fixated on a constrained search pretty quickly. Maybe it works pretty well, but I may have still missed something obvious ... or not so obvious.

The creative pattern matching that goes into finding a solution to a puzzler like this continues to be a mystery to me. I know my general strategy is to get the problem in my head and then leave the computer. I solve problems like this on walks or runs. I think one of the benefits of getting away from the keyboard is it takes away the ability to act on an approach too quickly. As I think back on the interviews I conducted, I think this is one of the big mistakes. The artificial time pressure and the inability to disengage from the social situation never allowed candidates a few minutes to just stare at the problems in their heads. In the end, this was a pretty fun puzzler even for someone that doesn't really enjoy puzzles. But from now on, I'm not reading any more startup blogs on my week off.

innerText vs. textContent

2013-02-27T00:00:00Z

Why does innerText require layout? I ended with that question in my previous post about layout thrashing. Just to recap briefly, there are very common patterns of use in the DOM APIs that cause terrible performance due to unnecessary layout. These posts highlight some of the oddities that I’ve found working on web performance for several years.

As with many other things in browsers, innerText’s behavior seemed to have happened due to overlapping (and, perhaps, under-defined) use cases. When you ask for the text contained within some DOM tree, you may be asking one of two questions: What is the raw textual content inside of these Nodes? Or, what is the text being presented to the user? These are similar, but obviously different. In a browser, textContent gives you the former and innerText the latter.

To illustrate some of the key differences, here is an example. Let us look at both innerText and textContent of the following HTML.

innerText

"lions, tigers"

textContent

"lions,\ntigersand bears"

Notice the differences, (1) the elements that are not rendered are also not present in innerText and (2) the line breaks in innerText follow the line breaks that were introduced by layout (not the original text we stuffed in the DOM). The best way to think about innerText is that it is roughly what you would get if you selected the text and copied. Whereas, textContent is just a concatenation of the values of all TextNodes in the sub-tree.

innerText is probably not what you want

The key takeaway is that innerText requires some information from the layout system to determine how the text is being presented to the user. This is what makes innerText one of those properties that can cause the performance of your app to go off the rails. Most libraries that favor innerText over textContent do so accidentally only because innerText was in Internet Explorer before textContent arrived as a properly specified API. For completeness, let me demonstrate the impact that choosing innerText over textContent can have on performance.

On a WebKit browser, you should see a significant performance difference (~300ms vs ~1ms). On IE9, you'll see better performance and a much smaller difference. It is clear that IE avoids computing a full layout and probably uses a special code path that computes only what is needed for innerText (which really isn't much). If you are using Firefox or Opera, you may be scatching your head. Keep reading.

While one could certainly conceive of use cases for innerText, most callers just assume that innerText and textContent are identical. You will see the expression node.innerText || node.textContent still being used in a number of libraries. Unfortunately, that leaves the door open for some unexpected performance problems. It is much wiser to prefer textContent these days.

Another reason innerText is probably not what you want

While it is still widely used, innerText is not standard. It is a bit of behavior that has lived on due to wide use during the Internet Explorer era. It's heavy use back then is probably the reason IE seems to have a specialized code path. To this day, it is not present in Firefox (wise decision on their part) and its behavior still varies widely in the browsers that do support it. Opera, for instance, merely computes textContent when you try to access innerText. This is why it outperforms WebKit in the example I show. When I use the expression "browser landmine", innerText is what I have in mind. To quote my good friend, Joel Webber, “it’s slower, but at least it doesn’t work as you would expect.”

Need help? While my colleagues and I build and validate our own web-based product, we are taking on some limited frontend work. If you could benefit from our help (especially with performance related issues), please do get in touch.

On Layout & Web Performance

2013-01-26T00:00:00Z

Outside of initial page load problems, “layout thrashing” is the most common performance problem I see in dynamic web applications. This is particularly true for Single Page Applications which build and destroy views on the fly. However, I’m often amazed by the number of web developers I run into who are unaware of the patterns that cause browsers to do unnecessary layout (aka “reflow” in the Mozilla community). And if you develop using a WebKit browser, there are plenty of tools available that will point out these problems.

Computing and invalidating layout

Like almost every other UI toolkit, your web browser has a notion of automatic layout where it takes the elements in a page adds the CSS styles and figures out where on the screen each element should appear. This can be an expensive process and, for obvious reasons, the expense grows proportional to the number of elements in a page. The part that gets developers in trouble is that web browsers use an on-demand model for layout. It avoids calculating the layout of a page until it really needs to know where on the screen something will appear. This, however, interacts with the DOM APIs in surprising ways. Consider this code:

Accessing offsetHeight is one those parts of the DOM API that requires the browser to have an up-to-date layout. It is important to realize that changing styles can invalidate all the work you just did so that asking for offsetHeight a second time requires another layout to be run. There are times when this is your intent, but far more often the second layout is completely wasted work. Compare, for instance, this similar snippet:

Because of this behavior, it is important for complex web applications to be mindful of when they are measuring (computing) and updating (invalidating) layout. In my simple examples it is very easy to spot the issue, but when a UI is built properly with decoupled components this can be a much more difficult task. Proper encapsulation of UI components actually tends to make debugging these issues even harder as each widget tends to take care of measuring the things it needs on construction to minimize the degree to which their abstractions leak.

Identifying layout problems

There was a time when finding these issues was very difficult. I once kept instrumented builds of all open source browsers and had a set of massive hacks for IE (that included preamble patching of native code). The situation in Chrome though has gotten a lot better. Fortunately, fixing layout issues in Chrome also tends to carry over to other browsers. My tool of choice for doing this is Speed Tracer. I’m biased, though, because I built it (with a lot of help, of course). There is also a panel built directly into Chrome’s Dev Tools called Timeline. The tracing data you find in Timeline and Speed Tracer is nearly identical since Timeline is actually built on the instrumentation we added for Speed Tracer. I still find Speed Tracer’s presentation of the data a lot more intuitive but, again, I did design most of it. If you can find your way around the Timeline Panel, it does have the benefit of being actively developed and gets regular feature updates. They recently added a Frame Mode, for instance, that organizes the timing data on a frame-by-frame basis.

Speed Tracer

Chrome's Timeline Panel

For instance...

It would be silly of me to talk about this problem without presenting an example. I have embedded a simple graph below and added two buttons that update some elements in that graph. The first uses a strategy that suffers from layout thrashing. The second does not. I will skip the detailed analysis in an effort to encourage you to try the tools I mentioned above. However, the code simply embeds the two patterns I described earlier.

Average yearly snowfall in various cities

Using Chrome on my own laptop, the first strategy takes about 250ms and the second only about 15ms. That is a pretty large difference given the two behave identically. As apps grow larger with more DOM elements and more styles applied to those elements, the issue worsens. I have actually seen very large apps taking seconds to do seemingly trivial updates due to layout thrashing.

What causes the browser to layout?

When developers realize this problem exists, they rightly want to know all the properties and functions in the DOM APIs that can cause layout. There are a quite a few. I assembled a list in 2010 for WebKit, which I have included below. A harder question to answer, however, is what changes in an element's style cause its layout to be invalidated. Answering that one often requires trial, error and good tools like the two I mentioned earlier.

Element

clientHeight
clientLeft
clientTop
clientWidth
focus
getBoundingClientRect
getClientRects
innerText
offsetHeight
offsetLeft
offsetParent
offsetTop
offsetWidth
outerText
scrollByLines
scrollByPages
scrollHeight
scrollIntoView
scrollIntoViewIfNeeded
scrollLeft
scrollTop
scrollWidth

MouseEvent

layerX
layerY
offsetX
offsetY

Window

getComputedStyle
scrollBy
scrollTo
scrollX
scrollY

Frame, Document & Image

height
width

Keep in mind that I only list HTML-related objects. I have intentionally ignored WebKit prefixed APIs and SVG objects, but many of those can cause layout as well. One surprising entry is Element.innerText. Maybe I can explain that one in another post.
[Update: innerText vs textContent]