FlightAware is a free flight tracker that will change what you think about live flight tracking and aviation data.

BSD Talk #42 has an interview Karl Lehenbauer about FlightAware.com. FlightAware tracks flight information, so far example here is their page on live flights to and from Sacramento Executive Airport. There is a lot of information that they are making available for free. Here is an outline of some of the more interesting bits that were mentioned by Karl during the interview.

All of FlightAware’s systems are 64-bit AMD based computers running FreeBSD 6.x, specifically the FreeBSD/AMD64 port.

They use PostgreSQL for the database back end. Slony 1 is being used to replicate data. Hard drives are in a RAID 1 (mirroring) configuration using 3ware controllers.

Now for some numbers:

• Receiving the data and processing it puts them about 6 minutes behind real time.
• Generating one map can be done in about 160 milliseconds of CPU time.
• Capable of generating several million maps a day.
• About 1 TB of stored data.
• Approximately 40 million position updates on air craft per day.

PostgreSQL wasn’t able to keep up with the updates so they wrote a memory resident database service queries. I’m still not exactly clear on what the relationship is between PostgreSQL and their memory resident database, which uses about 1 GB of RAM.

Nice to see a company putting FreeBSD and PostgreSQL to good use. I’m curious about the 40 million inserts per day number. Bring on the math!

• 40,000,000 inserts per day
• 40,000,000 / 24 = 1,666,667 inserts per hour
• 1,666,6667 / 60 = 27,778 inserts per minute
• 27,227 / 60 = 463 inserts per second

So that boils down to about 463 inserts per second on average. I’d expect that their actual peak requirements are much higher than that (perhaps two or three times that number?). That is just data that they are receiving, that doesn’t include the queries being run against their system to power the website. This brings up another question, how much bandwidth do they have dedicated to receiving these updates? It is possible that each individual update is fairly small (lat, long, src, dest, flight id, airline, plane type, etc) so that might not be too bad. Even at 256 bytes per update, doing 40 million of those a day adds up very quickly.

This will run faster in FreeBSD 7.0-RELEASE When it got released with ULE 2.0 which was done by Jeff Roberson a FreeBSD commiter