US NAVY using ADS-B code hopping

US Navy P-8 Poseidon taking off at Perth Airport Darren Koch (GFDL 1.2 or GFDL 1.2), via Wikimedia Commons

Recently, I started to detect many P8 Poseidon aircraft above South China Sea, like if a dozen of different P8 were flying in the area each day. The only explanation I see is that they are doing ADS-B code hopping, switching to a new TACTICAL code every few minutes.

Here are some of the codes I detected in the past few days: AE686A, AE6879, AE6889, AE67B3, AE6784, AE687D, AE6850, AE6854, AE6809, AE6793, AE6807, AE6845, AE6819, AE6836, AE67C2, AE67DC, AE6885, AE67D6, AE6813, AE687E, AE689E, AE6884, AE67DE, AE6803, AE682D, AE687C, AE6849, AE68A2.

Few more codes detected recently: AE6848, AE67DF, AE6870, AE6883, AE683D, AE6822, AE6814

3 tips to optimize PHP code

How I divided my server errors by 40 with PHP code optimization

I have been running several Raspberry Pi based trackers for a while, and I wanted to create a common database of detected aircraft and their positions. I looked for online database solution and how I could connect my Pi to them directly in Python, but that was not convenient.

I finally decided to get an hosting plan at OVH and create my own databases, with API to add more aircraft and positions. I wrote all the APIs of my server in PHP, as it was pretty straightforward. I got it work pretty well, but I was still having quite a lot of errors. At first, it was simple, I played a little with the timeout, I add retries, but I was still not able to go below 2% of errors. For a long time I lived with that, my system still worked well, but I finally decided to work on that and optimize PHP code.

The usual PHP tips to optimize PHP code…

A quick search on php optimization got me a list of things to do:

  • use single quote instead of double one: 'text' is better than "text"
  • calculate variable only once in a loop
  • close databases
  • order switch by order of use
  • select specific columns instead of * in mySQL SELECT queries
  • use foreach instead of while
  • use echo $a,$b,$c instead of echo $a.$b.$c
  • use sprintf to add strings: $string = sprintf('today is %s', $date)
  • use === instead of == in if: if ($a===$b) { action}
  • INSERT multiple values in database instead of multiple single INSERT

It seems that many of them were useful for older PHP versions, but not really for PHP versions over 7…

… and what really worked for me!

The 3 actions that had a tremendous effect on my API have been around database optimisation: first, I rewrote the few INSERT loops in order to INSERT multiple values at once. This was already much better. Then I checked my databases structures, to make sure I had all the right indexes corresponding to my SELECT queries. And finally, I switched all the heavy ones to InnoDB (I am not sure why OVH use MyISAM as default structure…).

These 3 actions reduced my error rate from 2% to 0.05% by reducing the load of my server (I am using a very basic hosted configuration, nothing fancy here). On the other tips I found, most of them had no or only very marginal impact. The only one I will keep, is the sprintf function to create my strings with variables as this makes my code much easier to read (it was easy to get lost in the quotes when you write mySQL queries with all that ‘ and ” and .).

How did I find that was my issues? What is important is to find where you PHP requests are spending time. For that, I simply implemented some time measurement inside the code (with the microtime function: $start_time=microtime(True); $stop_time=microtime(True); phpduration = $stop_time-$start_time;), that is coming back to my Pi with the result of the API. I could then dig into the one loading the server for more than 1s and reduced their load. I did the same in my Pi, to track which mySQL queries were taking more than 0.001s to run, and that was very efficient.

I was able to focus on optimising the most important ones (mainly those on huge database, such as positions, which rapidly contain millions of entries), and optimise their structure and indexes. There are many tools available to profile your PHP code, but I didn’t find any that was easy enough to start with compared to what I needed them for.

With 0.05% error rate, I can think about adding more trackers and maybe even have a way for others to share their tracking data with me! Let me know if you would share yours or if you have more tips to optimise PHP code…

What else can help reduce connection errors?

I also did several actions to reduce connection errors to my API server:
– make sure you don’t have any redirection
– make sure you updated requests and certifi on the Raspberry Pi, specially if you are using a secured connection. It can easily be done by:
pip install --upgrade requests certifi urllib3 idna chardet
– use session if you are calling the API regularly:
api_session = requests.Session()
resp = api_session.get([request])
will replace requests.get([request])
– set a different timeout for connection and response:
api_session.get([request], timeout=(5,10))

SigFox for Raspberry Pi

The hardware

First you need to find a Sigfox hat for your Raspberry Pi. Just Google “Sigfox for Raspberry Pi” and you will find the latest solution available. I am using a RPISIGFOX, made my SNOC. Pretty easy to setup and use. Just plug it in, change the serial port parameters, and it is ready to work. It comes with 1 year SigFox subscription that requires activation on Sigfox website. Depending on where you intend to use your Pi, you may need a different version of Sigfox radio (that one is mainly for Europe – US and Asia have different frequency bands for non regulated spectrum).


The Software

  1. Disable Raspberry Pi terminal on serial port with raspi-config utility: sudo raspi-config. Go to Interfacing Options then choose Serial then NO and OK
  2. Install pyserial: sudo apt-get install python-serial
  3. Download scripts: git clone https://github.com/SNOC/rpisigfox.git
  4. Enable script execution: chmod +x sendsigfox.py
  5. Plug antenna with its cable and send your first message: ./sendsigfox.py 0123

Setting up the Callbacks and your server

You also need to setup the callbacks on Sigfox system, and your server to manage the data. This is pretty straightforward with the documentation for the uplink messages. You are limited to 6 uplink messages per hour. On reception of the uplink message, the server will call your API, as you have defined in Sigfox interface. What you do with the data on your server is up to you!

Downlink messages are a little more complex to manage. You are limited to 4 downlink messages per day, and it was not very easy to make sure my Rasberry Pi application was remaining inside this budget, specially with a solar powered Pi and without a RTC … It took me some time to understand that the server was in fact the one making sure you stay in the budget, as if there is no downlink message to send, it won’t count into the 4 messages per day.

To request a downlink message, you just need to send ack=true with your uplink message.

On reception of the uplink request through the above API, your server will have to provide the answer: "[device ID]":{"downlinkData":"[data]"} if there is a downlink payload or "[device ID]":{"noData":true} when there is no payload to send. In that case, the downlink message will not count in your daily budget.

You can find more details on this great post on Sigfox downlink messages.

Keeping the Pi on time

As my Raspberry Pi is solar powered (with a 20W solar panel, which is enough for the Pi, but not for the RTL-SDR dongle that has a much bigger power consumption), I don’t necessarily have enough power to keep the Pi on 24/24. When it is off, the time stops to progress, as the Pi doesn’t have a RTC by default. After few days, time and date are completely wrong. That makes it difficult to control the downlink messages budget from the Pi. At the same time, SigFox is a great solution to get the network time to and update the time of the Pi. With the downlink messages, I was able to get the network time when the Pi is starting and update the date and time of the Pi with the simple command sudo date --set="9 JUN 2017 19:15:00".

How I use it?

With the payload limitations at 6 uplink messages per hour (12 bytes) and 4 downlink messages per day (8 bytes), Sigfox doesn’t really replace a Wifi connection, I cannot report all the detected aircrafts and their positions as I am normally doing. As, this is for an ADS-B tracker, I decided to focus on new aircraft and special ones (such as military) for the uplink direction, transmitting only the ModeS hex code and the timestamp at which it was detected, the remaining 4 bytes are used for maintenance purpose (HHHHHHYYYYMMDDHHMMSSXXXX).

To avoid going over 6 uplink messages per hour, each detection will go into a buffer, that is checked every 10 minutes. I have been working on some more compressed way to encode these data but practically I don’t need it, as I am very seldom over budget (meaning I am sending a message every 10 minutes as long as my Pi is powered by the battery). We will see if this is still the case after COVID-19 and the airlines start to have a more international flights. Each time my server receive an uplink message it will automatically generate a tweet (with the #Sigfox hashtag).

Downlink messages are used only to keep the time of the Pi in sync, as described before. As sometimes the Pi doesn’t receive the downlink message or is switched off again when battery is low and weather is grey, it may need more than one downlink per day for time sync. In addition of the network timestamp, I still have 2 bytes available to send other data to the Pi. I am using this for maintenance purpose (YYYYMMDDHHMMSSXX). Note that the uplink message must be exactly 8 bytes in size or it will not go through.

It has been a long time I wanted to play with a Sigfox hat, and it has been quite fun: not that difficult to make it work, but enough issues to solve to make that interesting!

Breitling Jet Team

Breitling Jet Team, operated by Apache Aviation is based in Dijon, a city in France few hundreds kilometres south east of Paris. My mobile tracker picked most of the team when I spent few weeks in Chalon our Saone early this year.

Breitling Jet Team is flying on Aero L39C Albatros, with tails ES-YLX, ES-YLN, ES-YLF, ES-YLP, ES-YLR, ES-TLF, ES-TLG and ES-YLI. You can see more details in the search page.

Breitling has decided in 2019 not to renew its sponsorship as sole and exclusive partner of Apache Aviation, but is open to staying on board as co-sponsor. Apache Aviation and Breitling are in search of co-sponsors to operate Jet Team around the globe.

Founded by Jacques Bothelin in 1980, the team had several sponsors (and names) during the years: Patrouille Martini, Patrouille Ecco, Patrouille Addecco, Khalifa Jet Team before to be named Breitling Jet Team in 2003.

Let’s hope they will find a new partner to continue this adventure!