SigFox for Raspberry Pi

The hardware

First you need to find a Sigfox hat for your Raspberry Pi. Just Google “Sigfox for Raspberry Pi” and you will find the latest solution available. I am using a RPISIGFOX, made my SNOC. Pretty easy to setup and use. Just plug it in, change the serial port parameters, and it is ready to work. It comes with 1 year SigFox subscription that requires activation on Sigfox website. Depending on where you intend to use your Pi, you may need a different version of Sigfox radio (that one is mainly for Europe – US and Asia have different frequency bands for non regulated spectrum).


The Software

  1. Disable Raspberry Pi terminal on serial port with raspi-config utility: sudo raspi-config. Go to Interfacing Options then choose Serial then NO and OK
  2. Install pyserial: sudo apt-get install python-serial
  3. Download scripts: git clone https://github.com/SNOC/rpisigfox.git
  4. Enable script execution: chmod +x sendsigfox.py
  5. Plug antenna with its cable and send your first message: ./sendsigfox.py 0123

Setting up the Callbacks and your server

You also need to setup the callbacks on Sigfox system, and your server to manage the data. This is pretty straightforward with the documentation for the uplink messages. You are limited to 6 uplink messages per hour. On reception of the uplink message, the server will call your API, as you have defined in Sigfox interface. What you do with the data on your server is up to you!

Downlink messages are a little more complex to manage. You are limited to 4 downlink messages per day, and it was not very easy to make sure my Rasberry Pi application was remaining inside this budget, specially with a solar powered Pi and without a RTC … It took me some time to understand that the server was in fact the one making sure you stay in the budget, as if there is no downlink message to send, it won’t count into the 4 messages per day.

To request a downlink message, you just need to send ack=true with your uplink message.

On reception of the uplink request through the above API, your server will have to provide the answer: "[device ID]":{"downlinkData":"[data]"} if there is a downlink payload or "[device ID]":{"noData":true} when there is no payload to send. In that case, the downlink message will not count in your daily budget.

You can find more details on this great post on Sigfox downlink messages.

Keeping the Pi on time

As my Raspberry Pi is solar powered (with a 20W solar panel, which is enough for the Pi, but not for the RTL-SDR dongle that has a much bigger power consumption), I don’t necessarily have enough power to keep the Pi on 24/24. When it is off, the time stops to progress, as the Pi doesn’t have a RTC by default. After few days, time and date are completely wrong. That makes it difficult to control the downlink messages budget from the Pi. At the same time, SigFox is a great solution to get the network time to and update the time of the Pi. With the downlink messages, I was able to get the network time when the Pi is starting and update the date and time of the Pi with the simple command sudo date --set="9 JUN 2017 19:15:00".

How I use it?

With the payload limitations at 6 uplink messages per hour (12 bytes) and 4 downlink messages per day (8 bytes), Sigfox doesn’t really replace a Wifi connection, I cannot report all the detected aircrafts and their positions as I am normally doing. As, this is for an ADS-B tracker, I decided to focus on new aircraft and special ones (such as military) for the uplink direction, transmitting only the ModeS hex code and the timestamp at which it was detected, the remaining 4 bytes are used for maintenance purpose (HHHHHHYYYYMMDDHHMMSSXXXX).

To avoid going over 6 uplink messages per hour, each detection will go into a buffer, that is checked every 10 minutes. I have been working on some more compressed way to encode these data but practically I don’t need it, as I am very seldom over budget (meaning I am sending a message every 10 minutes as long as my Pi is powered by the battery). We will see if this is still the case after COVID-19 and the airlines start to have a more international flights. Each time my server receive an uplink message it will automatically generate a tweet (with the #Sigfox hashtag).

Downlink messages are used only to keep the time of the Pi in sync, as described before. As sometimes the Pi doesn’t receive the downlink message or is switched off again when battery is low and weather is grey, it may need more than one downlink per day for time sync. In addition of the network timestamp, I still have 2 bytes available to send other data to the Pi. I am using this for maintenance purpose (YYYYMMDDHHMMSSXX). Note that the uplink message must be exactly 8 bytes in size or it will not go through.

It has been a long time I wanted to play with a Sigfox hat, and it has been quite fun: not that difficult to make it work, but enough issues to solve to make that interesting!

US military planes in South China Sea

There are a lot of US military activity in the area between Taiwan, Hong Kong and the South China Sea. Transports, SIGINT, reconnaissance, protection of US carriers in the area… mainly conducted by the US Air Force and the US Navy. Here is a list of planes seen in past few months (click on the link to see the radar map when the trackers got a MLAT position – these aircraft generally do not broadcast their position on ADSB):

AE049B – Tail number: 61-0313 – Boeing KC-135R – US Air Force

AE1492 – Tail number: 96-0042 – Boeing E-8C Joint Stars – US Air Force

AE1D8A – Tail number: 15-6511 – Lockheed EP-3E Aries 2 Orion – US Navy

AE1D95- Tail number: 15-9893 – Lockheed EP-3E Orion – US Navy

AE1D91- Tail number: 15-7318 – Lockheed EP-3E Aries2 Orion – US Navy

AE1D5D – Tail number: 16-1586 – Lockheed P-3C-II.2 Orion – US Navy

AE17EF- Tail number: 05-0730 – Boeing C-40C – US Air Force

AE04FD – Tail number: 58-0095 – Boeing KC135T – US Air Force

AE01CE – Tail number: 62-4134 – Boeing RC135W Stratotanker SIGINT – US Air Force

You also can see the latest detected military and other aircraft of interest and the pick of the day or search my database on foxtrotcharlie.ovh.

Having fun with my ADS-B tracker project

How it started

Some years ago, I bumped into an article on Mashable about Jeremy Merrill using a Raspberry Pi ADS-B tracker to detect aircraft flying over his house and display the origin or destination of that plane (see article here and github there). As my place is just on a busy plane corridor, that gave me an idea. I decided to work on a similar project based on Piaware. Looking at how he did it and at the long literature you can find on the web on ADS-B, I created this site, my own software in Python to run on my Pi and other fun stuffs. I also ran into this site by SonicGoose and that gave me many new ideas (including the basic structure of this site – I never had any experience of html or php before).

One key part is getting access to aircraft databases to get more info from the ModeS hex code you get from the Piaware tracker. I am sharing my own database of planes, airports and routes I detected with one of my trackers (I have a fixed one at home, another one in Briancon in France (solar powered and connected by SigFox!) and a mobile one I take with me during my trips over the world).

When I started few years ago, I didn’t know that it will become such a fun project!

Updates

[2019/02] Update of installation script and new links
[2019/05] Update of the Radar page that now use my own positions database. Added aircraft silhouettes and Airlines logo.
[2019/05] Changed the domain to FoxtrotCharlie.ovh !!!
[2019/06] Changed landing page and reworked index for Google
[2019/07] Added Great Circle tracks between origin, destination and seen positions on Radar
[2019/09] Added Airports data, API page, better sitemap for Google index
[2020/01] Added a solar powered ADS-B tracker in Briancon, with a SigFox interface by SNOC to send new / special planes to this database when Wifi is off – works great!
Running an ADS-B tracker on a solar panel is quite challenging. After some research, I believed a 20W solar panel would be enough, but the stick consumption is very high, and solar energy production is far from stable… after few month using it, I would need about 80W to be able to power the system for 24h (Raspberry Pi 4 + Flightaware Pro Stick Plus + SNOC SigFox). SigFox is a great solution to replace a 4G modem which would also take too much energy. The limitation to 6 messages per hour is manageable, as I am only reporting new or special planes to this website, while the detailed data are stored and shared when the device has access to wifi.
The biggest issue I didn’t plan for was that the Pi doesn’t have a RTC, so time was wrong (it only progress when the Pi is on, about 8 to 12h per day… after few days, it was completely off). Hopefully, I could also use the downlink SigFox messages to get the network time and update the Pi time at each boot. Here the limitation is at 4 messages per day. It is controlled by the callback feature so it is quite easy to be sure you will not exceed the budget, even with a Pi with no idea what is current date or time. Next step is to work on shuting down properly the Pi when battery is low in order to avoid memory issue. So far it works with multiple wild shutdowns per day, but I don’t know for how long.
[2020/02] Updated the stats page to show more details
[2020/03] Added tweets on Sigfox interface with Briancon tracker
[2020/07] Working on adding MoPi-2 on the solar powered tracker to allow clean power down. Changed all the time in UTC. With trackers everywhere, it doesn’t make sens anymore to be on Hong Kong time.
[2020/09] Added a new blog. Let’s hope I will post regularly from now on![2020/10] API optimisation – reduced the errors from 2% to 0.05%
[2020/11] New landing page with a RSS feed reader to get up to date links to my blog posts. Let’s see the impact on Search ranking!
[2020/12] Upgraded all my Raspberry Pi code to Python3. I have been delaying that for a long time, but it was much easier than expected. Further optimisation of the API to reduce the error rate further. Some trackers are now at 0 errors, some still at 0.05%, still struggling to understand why.

Interesting links on ADS-B trackers

Sharing my database of Aircraft, Airports and Routes

I have been gathering data for few years now, from multiple sources (databases found online, some special aircraft found manually online, …). I now have more than 32,500 entries in my Aircraft database, including 6,000 that I flagged as “special”, meaning military or any other interesting features (helicopters, rare aircrafts …).

I also have more than 16,000 airports and 160,000 routes.

They are all accessible through dedicated API.

Search aircraft by Mode S code or tail number

You can access my database of aircraft through the following API:
https://www.foxtrotcharlie.ovh/API/public/api?ModeS=XXXXXX
where XXXXXX is the ModeS code of the aircraft, on 6 digits in hexadecimal.
https://www.foxtrotcharlie.ovh/API/public/api?Registration=XXXXXX
where XXXXXX is the tail number (or registration) of the aircraft (in some countries, tail numbers have a – like in China B-8638, it should be put here).
The output is in json format (RFC4627).
Working samples of the API:
https://www.foxtrotcharlie.ovh/API/public/api?ModeS=78102f
https://www.foxtrotcharlie.ovh/API/public/api?Registration=B-8638

Returned data:

      {
          "ModeS":            "78102f",
          "ModeSCountry":     "China",
          "Registration":     "B-8638",
          "ICAOTypeCode":     "A321",
          "Type":             "Airbus A321-211",
          "SerialNo":         "---",
          "RegisteredOwners": "China Southern Airlines",
          "OperatorFlagCode": "CSN",
          "FirstSeen":        "2017-03-25 23:58:09",
          "LastSeen":         "2019-09-06 07:25:41"
      }
  
DesignatorDescriptionFormat
ModeSModeS code in HEX format of the Aircraft6 characters
ModeSCountryCountry of registration of the Aircraftstring
RegistrationTail Number of the Aircraftstring
ICAOTypeCodeICAO Type Code of the Typestring
TypeType of Aircraftstring
SerialNoSerial Number of the Aircraftstring
RegisteredOwnersRegistered Owners of the Aircraftstring
OperatorFlagCodeCode of the Airline if anystring
FirstSeenTime stamp when my trackers first detected the AircraftTimestamp
LastSeenTime stamp when my trackers last detected the AircraftTimestamp
Json structure description for Aircraft API

Search airports by ICAO code

You can also access my database of airports through the following API:
https://www.foxtrotcharlie.ovh/API/public/api?ICAO=XXXX
where XXXX is the ICAO code of the airport on 4 characters.
The output is in json format (RFC4627).
Working samples of the API:
https://www.foxtrotcharlie.ovh/API/public/api?ICAO=VHHH

Returned data:

      {
          "ICAO":       "VHHH",
          "IATA":       "HKG",
          "Name":       "Hong Kong International Airport",
          "Location":   "Hong Kong",
          "Region":     "Hong Kong",
          "Latitude":   22.3089,
          "Longitude":  113.915
       }
DesignatorDescriptionFormat
ICAOICAO code of the Airport4 characters
IATAIATA code of the Airport3 characters
NameName of the Airportstring
LocationLocation of the Airport in clearstring
RegionRegion or Country of the Airportstring
LatitudeLatitude of the Airportfloat
LongitudeLongitude of the Airportfloat
Json structure description for Airport API

Search routes by Flight Number (ICAO format)

You can also access my database of routes through the following API:
https://www.foxtrotcharlie.ovh/API/public/api?FNB=XXXXXX
where XXXXXX is the flight number in ICAO format.
The output is in json format (RFC4627).
Working samples of the API:
https://www.foxtrotcharlie.ovh/API/public/api?FNB=AFR188

Returned data:

      {
          "FNB":            "AFR188",
          "Airline":        "AFR",
          "FlightNumber":   "188",
          "Origin":         "LFPG",
          "Destination":    "VHHH",
          "Via":            ""
       }
DesignatorDescriptionFormat
FNBFlight number in ICAO formatstring
AirlineICAO code of the airline4 characters
FlightNumberFlight numberstring
OriginOrigin airport (ICAO format)string
DestinationDestination airport (ICAO format)string
ViaVia airport – if any (ICAO format)string
Json structure description for Airports API

Disclaimer

All the information provided by these API are provided as is and for information only. They are coming from different sources and may contain mistakes or be out of date. Please use with caution and at your own risk.

You are welcome to comment if you see mistakes. Mode S codes can be re-allocated to new aircraft, and keeping the database up to date is not an easy task.

G-IRTY Spitfire flying over Hong Kong in 2019

One of the most surprising catch of 2019 was for sure seeing a Spitfire in my new aircraft list! I had to check if that was correct or a mistake somewhere in the database of Mode S codes I collected.

It turned out it was the real thing! The Silver Spitfire was doing The Longest Flight, for the 75th anniversary of the D-Day invasion. A 30 legs, 22,138 miles trip around the world.

These guys rock!

A Guide to Aircraft Tracking Around the World

If your a fan of Aircraft Tracking, I highly recommend the article from Global Investigative Journalism Network (or GIJN) “Planespotting: A Guide to Tracking Aircraft Around the World“. This is one of the most comprehensive article I have seen on the subject, addressing it from multiple angles (#avgeek, journalist, activist, …). They even propose a short version if you just want to start tracking.

Great work!

Another F16 from Belgian Air Force for my tracker in Briancon

My tracker in Briancon has detected another F16 from the Belgian Air Component recently (that’s how the Belgian Air Force is called).

The aircraft detected so far have the tail numbers FA-87, FA-57, FA-133, FA-130, FA-101 and FA-72.

Belgian F16 are regularly flying over France.

ADS-B tracking with an off grid Raspberry Pi

I have been looking for a solution to do ADS-B tracking off grid for quite a while. I have several Piaware trackers in different setup (fixed one at home, mobile one with GPS in the car, mobile one that I take with me when I travel abroad), and I wanted to add some in places where I don’t have power or wifi. After some research I decided to use a solar panel for power and a SigFox hat for connection. The SigFox hat has limited data throughput (6 messages per hour in uplink), but it is enough to report new or interesting aircraft to my server.

There is a lot of literature on solar power for the Pi, but I still ended up underestimating the size of the panel. 20W just gives me 8h per day in average. I will need to upgrade to 100W to make sure I run 24/24 7/7 365/365. The BIG power consumption comes from the RTL-SDR dongle, and you can not toggle it on and off to save power as it needs to be on for an extended period of time to do its job. Having my tracker going on and off created an unexpected issue that I didn’t see during my debug session with wifi: as the Pi does not have a RTC, the Pi time is losing 16h per day in average.

Rapidly, my timestamp for the planes detected were off. Hopefully, as SigFox allows also 4 downlink messages per day, I am able to request a time update through this channel. With some adjustment to make sure I don’t use my downlink messages budget when the Pi powers up just for few minutes when battery is low but sun is high, I was able to make that work. Next step is to work on improvements for the battery management and I will have a fully independent solution. I now have a Piaware tracker running fully off grid!


You can find more details on my setup and the data reported by my trackers on https://www.foxtrotcharlie.ovh


When I started this project I didn’t know how far it would bring me! My long term goal is still to share all my software on Github, but that will need some cleaning!

Also published on www.raspberrypi.org

How to build an ADS-B tracker

In order to build your ADS-B tracker you will need just few items: a Raspberry Pi, a RTL-SDR dongle, an antenna, and some free time! For the tracker, I am using a Raspberry Pi.

I started with the A version, but it was a little limited (it was often at 100% CPU load, running dump1090, Piaware and my own software). I then moved to the B+ version, and I was down to 10% which was much better. I now run a Raspberry Pi version 4.

To setup your tracker, follow these steps:

WARNING: AS THINGS MAY EVOLVE, YOU MAY NEED TO ADAPT THE TUTORIAL TO YOUR HARDWARE AND SOFTWARE VERSIONS

– First you need to download the latest NOOBS from Raspberry Pi website, onto a SD card formated in FAT (your Raspberry Pi is not recognizing FAT32 format). I do recommand to use minimum 32 GB of flash, specially if you want to have some space for recording positions of the aircraft you will track. If not, 8 GB is enough.

– Insert the SD card into the slot on the Raspberry Pi and power the board. Then follow the instructions. For this step, you will need a keyboard, a mouse and a TV with HDMI input.

– When it is done, upgrade the Raspberry Pi software and firmware:

        sudo apt-get update
        sudo apt-get upgrade
        sudo rpi-update

– Install tightvnc:

        sudo apt-get install tightvncserver
        tightvncserver

– You now have a working Pi, that you can manage from a remote computer with a VNC software (I am using tightvncjviewer.jar on Mac)

– Next step is to install the ADSB tracker. For this, you will need a SDR (Software Define Radio) dongle, that you will plus to your Pi. I am using both that one and Piaware Pro-stick. The pro-stick has an integrated amplifier that will give you some additional range. Now there is a new version with integrated filter which is working great. I recommend to switch to one of the Piaware Pro-sticks as they consistently show better results than other sticks that I have tried over the years.

Install Piaware – that will give you a free access to cool features from Flightaware at the same time. You can first create credentials on Flightaware, to get a USERNAME and a PASSWORD.

        #install piaware
        wget https://flightaware.com/adsb/piaware/files/packages/pool/piaware/p/piaware-support/piaware-repository_4.0_all.deb
        sudo dpkg -i piaware-repository_4.0_all.deb
        sudo apt-get update
        sudo apt-get install piaware
        sudo piaware-config allow-auto-updates yes
        sudo piaware-config allow-manual-updates yes
        sudo apt-get install dump1090-fa

– You now have a working tracker and you can see your data on Flightaware. This is already quite fun.

See what else you can do from there on foxtrotcharlie.ovh.