The goal of this document is specifically to be very “nuts and bolts”, to avoid getting bogged down in all the context and to be prescriptive, i.e. “do this”, as much as possible, versus delving into context and basckground.
Despite my high ambitions, only a little of the info on this page is purely about writing pymavlink apps, a lot of it is about all the bits and pieces in the context. This is part of what makes this topic tricky, pymavlink doesn’t exist in a vaccuum, it doesn’t make sense to talk about it divorced from all the other elements. Sorry.
This document assumes you’re a smart, competent programmer and that you are basically familiar with MAVLink, know what it is and what it’s used for, and so forth. Or that you’re smart enough to google on words you don’t understand. I will try to give the bare minimum of context, to facilitate that googling, without (too late) getting too sidetracked into the sprawl that this topic in general tends towards.
You can disagree with what I’m suggesting, and in fact, I would gladly welcome feedback from people who know more about this than I do. I’m an experienced programmer, but definitely a MAVLink and pymavlink beginner.
However, despite my lack of mavlink/pymavlink experience, in order to make this easier to read I’m going to phrase things very confidently and authoritatively. Otherwise it’ll be twice as long and half as clear. All the extra language will be a bunch of hemming and hawing and it’ll be a pain to read.
This is all written assuming you’re working on Linux, because that’s what I’m working on, and Linux seems to be what most mavlink stuff and drone stuff is done on. It’s probably not impossible to write pymavlink code on Mac or Windows (though I suspect you’ll have more luck on Windows by using WSL), but there might be a variety of gotchas.
I know that, inevitably, I’m going to think of all sorts of random stuff that I want to throw in here, stuff that is off target from my stated “nuts and bolts” goal above. So, as those crop up, I’ll move them to the end of this document, in the section titled “Random Mavlink/Pymavlink/Ardupilot Links”.
Note, properly speaking it’s MAVLink, “Micro Air Vehicle Link”, but from here on I’m simply going to call it mavlink, because I’m lazy.
Okay, I know I said I want to get directly to the nuts and bolts, but there’s some relevant context that you do need to understand. If you want, you can come back and read this later (but do come back and read it later) and for now just skip down to “High Level Real Code”.
No piece of this exists in a vaccuum, so we can’t talk about any of it without talking about the other pieces, and how they all fit together.
One of the things that makes the documentation of all this “challenging” is that there are really always at least three things in the whole picture any time you’re talking about using pymavlink:
Pymavlink’s implementation of various mavlink messages.
The mavlink spec’s definition of the mavlink messages.
The flight controller’s implementation, in this case ArduPilot’s, of the mavlink messages.
Note, while in theory the mavlink spec comes first, in practice the spec defines the format for messages in between the pymavlink implementation and the ardupilot implementation, which is why I’m putting mavlink in the middle of the list.
Pymavlink is now part of the ArduPilot project and the repo is at:
https://github.com/ArduPilot/pymavlink
However, to see the pymavlink implementations you need to look in the dialect modules, and those don’t seem to be actually in the repo. The main place I’ve found them is by actually installing the pymavlink library and looking in the installed library source (see below “Pymavlink Dialect Code”).
The mavlink spec xml file is at:
https://mavlink.io/en/messages/
But note that it’s not just one xml file, they have the miminal, standard and common sets of entities, so for example you have:
https://mavlink.io/en/messages/common.html
The ardupilot docs don’t talk about pymavlink, they talk about mavlink messages as ardupilot implements them, and they have different doc pages for each ardupilot mode (copter, plane, rover, etc):
https://ardupilot.org/dev/docs/mavlink-commands.html
And also:
https://github.com/ArduPilot/ardupilot
There’s also the Dronekit project, which is sadly, as of this writing, rather languishing. That sucks, because dronekit aims to do exactly what I want, which is a higher level API, so you spend less time with the innards of mavlink. As I said, Dronekit is not getting a lot of love, but it’s probably still worth looking into, to better understand mavlink:
Dronekit-python is a little more active.
https://github.com/dronekit/dronekit-python
Pymavlink is a python library for writing mavlink apps.
Andrew Tridgell, the original author of pymavlink, wrote mavproxy, both back in 2011. In fact he wrote pymavlink because he needed pymavlink to write mavproxy:
https://diydrones.com/profiles/blogs/python-mavlink-interface
So mavproxy is the flagship application for pymavlink, therefore examining the source code of mavproxy is probably a good way to learn about pymavlink. I should do that, sometime.
Both pymavlink and mavproxy are currently officially supported and maintained by the ArduPilot open source project (of which Andrew Tridgell is one of the leaders):
https://github.com/ArduPilot/pymavlink
Also, the ardupilot website has a bunch of autotest scripts. To quote ArduPilot contributor Peter Barker:
In general, “how do I do X via mavlink” should be answered in the autotest suite. Big emphasis on the “should”, ’though.
https://ardupilot.org/dev/docs/the-ardupilot-autotest-framework.html
https://github.com/ardupilot/ardupilot/blob/master/Tools/autotest/arduplane.py
The ArduPilot project’s official docs for pymavlink do have some useful information, but I didn’t find them very accessible on my first reading:
https://mavlink.io/ko/mavgen_python/
When I revisited them later, after learning more about pymavlink and mavlink, I found the official docs more useful. I think the biggest problem is that they lack a conceptual overview, a structure or context to fit the different pieces of useful information into.
Yes, this is the background I wanted to avoid, but this part is nuts and bolts relevant, so:
The mavlink messages are defined in the MAVLink XML specification, and pretty much everybody just uses that XML specification to generate the bulk of the code to do the actual mavlink messages.
Side note: If you’re curious, the pymavlink code to generate the mavlink message classes appears to be here:
https://github.com/ArduPilot/pymavlink/blob/master/generator/
This generated code nature leads to some obvious, if you think about it, patterns of results with respect to libraries and documentation (to wit, they kinda end up being very low level, without very good documentation). I can’t blame them, I’d probably do the same; Laziness is one of the three cardinal virtues of a programmer.
Mavlink has dialects so pymavlink has dialect modules for each dialect, in addition to the different mavlink versions (1 and 2), and the “common” commands (which seem to be an intersection of all the mavlink commands that most flight controllers implement),
The different dialects seem to be specific to different flight controller hardware/software. If you think about it, this makes a lot of sense – exactly what the flight controller software does with the messages depends on each flight controller project’s interpretation of what it means and how it should work.
Ardupilotmega is the default dialect module in pymavlink. You can overide that a couple different ways, which are covered in the official docs and I’ll repeat them further down, but for the purposes of this doc we’ll assume you’re using the ardupilotmega dialect.
The dialect module is important to know about, because when you’re using pymavlink it will load the generated code from a mavlink dialect module, and that’s the code that will actually be run.
And the generated code isn’t in the pymavlink github repo – if you want to see the actual source of the actual code for sending the actual messages, you need to dig into the generated code for the dialect module, the first step of which is finding the generated code. I’ll get into the nuts and bolts of that further below, in “Pymavlink Dialect Code”.
Finally, mavlink has what mavlink calls mavlink microservices, which have nothing to do with microservices in the now-popular programming sense of the term.
Mavlink microservices are protocols in the higher level sense of the term, sequences of mavlink messages back and forth between the mavlink client program and the flight controller, necessary to complete certain tasks. What you might call a dialog of commands and responses between the GCS or your script, and the flight controller.
Here’s the official mavlink microservice doc:
https://mavlink.io/en/services/
Note, the mavlink docs for microservices (link below) also call most of the microservices “protocols”. So you might see somebody refer to, for example, either the mission microservice, or mission service, or mission protocol.
The most obvious microservice example I can think of, offhand, though I’m not sure this is an “official” mavlink microservice, is that if you want your drone to fly somewhere, you need to send it a sequence of mavlink messages:
Each of these messages will cause a response message – usually an ACK that specifically identifies what kind of message it’s responding to – and of course strictly speaking, your code should be looking for those responses before moving on to the next command in the sequence.
Again, these mavlink microservices aren’t well documented in most sources, but be aware that they’re out there. Hopefully, as this document gets better, I’ll add some info on the more frequently used mavlink microservices.
The ArduPilot website has several distinct pages for subsets of the mavlink commands, for different purposes.
These might be the same as mavlink microservices aka mavlink protocols. If not, then I’m not aware of a specific term for these subsets as a category, but they seem useful to know about.
So far mainly this is about the Mission commands (see the next section).
I know there are some other command subsets, for example there are three different subsets of commands related to “driving” the drone, unless maybe you don’t count the one set that’s about waypoints (i.e. set a target GPS coordinates and tell it to go there).
There’s another subset of commands to drive it like a car (point it in a direction and set speed) to support handset joysticks. That’s probably the “Manual Control Protocol”, which the mavlink site also refers to as the “Manual Control (Joystick) Protocol”.
https://mavlink.io/zh/services/manual_control.html
There’s also one that lets you directly set motor speeds and so forth. That’s probably not used so much with quadcopters (which tend to fall out of the sky when the motor speeds aren’t just right), more with ArduRover or ArduSub (which just sit there, or float there, until the motor speeds are set).
The section of the ArduPilot docs that encloses the messages linked below says this is more about Mission Planner than it is about mavLink, but it’s still getting into the mavlink commands.
These commands are all only available when the drone is in Auto mode, but that doesn’t necessarily mean they’re the only commands available in auto mode. I don’t know what else is, but don’t assume.
https://ardupilot.org/plane/docs/common-mavlink-mission-command-messages-mav_cmd.html
https://ardupilot.org/copter/docs/mission-command-list.html
I’m putting this up here near the beginning because, fundamentally, the code is the ultimate source of truth, so it’s a good idea to know where to find the code.
Unfortunately, as I said above, the dialect code is generated from the mavlink XML spec, which means you can’t find the dialect code in the github repo. For example, if you look at the modules directory in the pymavlink github repo:
https://github.com/ArduPilot/pymavlink/tree/master
And then try to dig down into the dialects code:
https://github.com/ArduPilot/pymavlink/tree/master/dialects/v10/python2
…it’s empty.
If you want to look at the dialect code, you need to install the python pymavlink library and look in there. On my system I’m doing this all in a pymavlink_testing directory in my home directory, and I used python’s venv to create a venv named mav_venv, and installed pymavlink in mav_venv, so I find the mavlink dialect code in:
/home/puff/pymavlink_testing/mav_venv/lib/python3.8/site-packages/pymavlink/dialects/v20/ardupilotmega.pyThe dialect source code also has help docs (yay) so you can use that and python’s help function and the python CLI to learn more about the code.
For whatever reason, there doesn’t appear to be an “official” web page documenting the all of the dialect module message methods, or at least I haven’t been able to find it.
Note: “module message methods” is really not a great phrase, because pymavlink also has message objects, which of course have methods. For now I’ll stick with “module message methods” while I try to think of a better phrase.
However, while I’m still not sure of the backstory of the following link (i.e. where it came from, who made it, ec), these are some somewhat useful API docs for pymavlink:
https://www.samba.org/tridge/UAV/pymavlink/apidocs/
I might get around to writing a script to mine the help docs and add them to this doc (or to a separate web page). But don’t hold your breath.
Finally we’re going to get real here. Well, as real as code ever gets.
There are two halves (or at least two big chunks) to pymavlink, the dialect module that I describe above, and mavutil, which is lower level.
I’m pretty sure the dialect module uses mavutil to get stuff done. mavutil is a module, you can access members directly, I’ll give an example below.
For most (most? maybe all?) of the other stuff you have to create and set up a mavlink connection object, which in all of my examples in this doc I’ll assume has been set up and stored in a variable named mav_con.
For example here we’re passing the data from a mavlink message into mavutil.all_printable() to… well, print all the data:
mavutil.all_printable(msg.data)You can access the dialect module message methods via the mavlink connection object, specifically from the connection object’s mav member variable, for example:
mav_con.mav.message_method_name()Note: Of course, you first have to create the connection object and then the mavlink session, that’s described below in “Down To Brass Tacks, er, Nuts and Bolts”.
Reportedly it’s not uncommon for pymavlink programmers to use mavutil for a lot of stuff, though I wonder how much of that is because of how hard it is to find the dialect module docs.
Pymavlink has a whole bunch of low-level code that you can use to construct and send mavlink messages, but probably shouldn’t. It’s a lot easier to use the higher level module message methods, which are in the mavlink dialect module.
Here’s an example of how you’d set guided mode, using the low level pymavlink code. Note that this is actually accessing the dialect module’s implementation of command_long_send() method via mav_con.mav.
It’s hard to get this code to format well in markdown, so in both examples I’ve taken out one of the uglier-to-format bits and put it on a separate line. Specifically, I’m getting the mode flag value for custom mode, and sticking in a relatively shorter-named variable.
Then the second line of each example uses that mode flag variable.
custom_mode_enabled_flag = mavutil.mavlink.MAV_MODE_FLAG_CUSTOM_MODE_ENABLED
mav_con.mav.command_long_send(mav_con.target_system,
mav_con.target_component,
mavutil.mavlink.MAV_CMD_DO_SET_MODE,
0,
custom_mode_enabled_flag,
mav_con.mode_mapping()["GUIDED"],
0, 0, 0, 0, 0)And here’s how you’d do it via the higher level dialect module message method, set_mode_send().
custom_mode_enabled_flag = mavutil.mavlink.MAV_MODE_FLAG_CUSTOM_MODE_ENABLED
mav_con.mav.set_mode_send(mav_con.target_system,
custom_mode_enabled_flag,
mav_con.mode_mapping()["GUIDED"])Note that we didn’’t need to include parameters for target_system or target_component in this version. I’m pretty sure those get pulled from mav_con, which in turn got them from the connection setup process, probably from the heartbeat message it received from the drone.
Pymavlink does not necessarily have module message methods for all message types. For example, for arm and disarm, I don’t see anything in the dialect code, so you just have to do:
master.mav.command_long_send(1, 0,
mavutil.mavlink.MAV_CMD_COMPONENT_ARM_DISARM,
0, 1, 0, 0, 0, 0, 0, 0)Lets start off by listing several important pymavlink modules/classes/methods/etc that you’ll be using eventually:
# All the members on the mavutil module.
mavutil.*
# How you create a mavlink connection object,
# and then use the wait_hearbeat() blocking call, which establishes
# the mavlink session in your mavlink connection object
mav_con = mavutil.mavlink_connection(connection_string)
mav_con.wait_heartbeat()
# call this to receive mavlink messages
mav_con.recv_match()
# a dict of the last received message of each type
# keyed by message type, of course.
mav_con.messages
# last time this message type was received
mav_con.time_since('MESSAGE_TYPE_STRING')
# the dialect module message methods
mav_con.mav.some_message_method_name()
# to send the heartbeat from your pymavlink script:
mav_con.mav.heartbeat_send(mavutil.mavlink.MAV_TYPE_ONBOARD_CONTROLLER,
mavutil.mavlink.MAV_AUTOPILOT_INVALID,
0, 0, 0)
# the mavlink common ENUMs
mavutil.mavlink.SOME_ENUM_NAME
# the mavlink dialect module ENUMs
mav_con.mav.SOME_ENUM_NAME Generally speaking, the outline of the pymavlink part of your program will be this:
The first thing it will do is get a mavlink connection by calling mavutil.mavlink_connection(connection_string)
The second thing it will do is establish a mavlink session by calling mav_con.wait_heartbeat().
Then third thing it will do is start your main loop. Your loop will do a short list of things automatically and frequently:
At least once every second, your loop will call mav_con.heartbeat(send), or the drone will assume the session was dropped.
Your loop will regularly call mav_con.recv_match(), even if it doesn’t need to receive any mavlink messages (i.e. it ignores and discards the messages), to clear the network buffer and allow new mavlink messages in.
Your loop will send any mavlink commands it needs to by calling mav_con.mav.some_message_method_name()
Your loop might also need to call methods on mavutil or it’s instance members.
Your loop will check recv_match() output for ACK messages to its mavlink commands.
Your loop might check various things about last known state of the drone by looking in the dict in mav_con.messages, which contains a dict of the last received message of each type.
Your loop also has to do whatever other logic you want it to do, i.e. whatever logic you need to do in order to decide what mavlink messages need to be sent.
Pymavlink is not itself multithreaded and it also does not support asyncio. Depending on the complexity of your logic or your app, you might want to run your pymavlink loop in one thread and have the logic of the program run in another, and have the logic thread communicate with the pymavlink loop thread via a pair of thread-safe queues.
Someday I’ll add an example of how to do that.
You get a mavlink connection object and establish a mavlink session by doing this:
mav_con = mavutil.mavlink_connection(connection_string)
mav_con.wait_heartbeat()The connection_string is similar to a URL, in that it tells mavlink where to get the mavlink messages. Commonly this is either a serial port, in which case the string is just the path to the serial port (for example /dev/ttyACM0), or it’s a network connection (for example udpin:0.0.0.0:5761). I’ll talk more about that below, under the section titled “MAVLink Connection String”.
Note that wait_heartbeat() is a blocking call, so if you’re doing python asyncio stuff, this can hang up your entire app. Pymavlink has no asyncio support, by the way. However, many of the mavlink calls (other than wait_heartbeat()) are just formatting and sending a UDP packet, and thus are usually fast enough that it doesn’t seem to cause problems with asyncio.
Note that your script also has to send a heartbeat, at least 1Hz (one per second), or the flight controller will decide you’ve lost connection and might execute some failsafe behavior, like landing immediately (which, depending on what’s below it, could be… bad), or going “home” and landing (which, depending on where it thinks “home” is, could be… bad).
See the section “MAVLink Connection String” for an explanation of what mavlink connection strings are and where they come from.
You receive messages from the drone by calling mav_con.recv_match().
You can also call the recv_match() method with a type="MESSAGE_NAME_STRING" argument to filter for specific message types.
Note that recv_match() may not return a valid message. You should always test the return value. For example here:
msg = mav_con.recv_match(type="GLOBAL_POSITION_INT")
if not msg:
# got a "falsy" value (e.g. None or similar),
# ignore it,
# and/or return an error code to whatever called this code,
# and/or log it, whatever
elif msg.get_type() == "BAD_DATA":
# got a message object, but it's bad data,
# ignore it, return an error code, log it, whatever
else:
# hey it's a good message, do something with it!From the official docs at:
https://mavlink.io/ko/mavgen_python/
The returned object is the subclass of MAVLink_message for the specific message.
[…]
If needed, you can query MAVLink_message for information about the signature, CRC and other header information.
So the message superclass is MAVLink_message, and I think (based on the above) that means you can check inherited fields from the MAVLink_message class.
Pymavlink is at least a little stateful.
First, you must call the recv_match() method on a regular basis to empty it, even if you don’t need the messages in it, or later messages will get dropped.
I’ve seen people refer to recv_match() reading from some sort of queue, but according to this post, behind recv_match() in pymavlink is a buffer that contains packets containing mavlink messages, not a queue or a list:
Don’t quote me on this, but it could just be that the “buffer” or “queue” people are talking about is the OS level network stack for receiving packets.
Note, that post has a lot of good detail on how recv_match() works, I highly recommend reading it.
Second, the connection object has an instance variable named “messages” that contains a dict of the most recent of each message type that it has received.
Here’s an example of using the messages dict:
try:
gps_msg = mav_con.messages['GPS_RAW_INT']
altitude = gps_msg.alt
timestamp = mav_con.time_since('GPS_RAW_INT')
except:
print('No GPS_RAW_INT message received')If your pymavlink script is talking to the drone via a serial port, the connection string is the path to the serial port, something like: /dev/ttyACM0
If your pymavlink script is talking to the drone via wifi or some other flavor of Internet, the connection string will vary depending on your configured protocol, usually UDP, something like: udpin:0.0.0.0:5761
Typically, your pymavlink script is running in one of two contexts:
On a companion computer, which is part of the drone, and physically connected to the flight controller via a serial cable plugged into the serial port (or sometimes via serial-over-USB cable, but a real serial cable is generally recommended).
On a ground station computer (aka laptop, etc) connected via a pair of radios that make it look like it’s physically connected via serial port.
One of the more common patterns of using radios to control drones is to use a pair of telemetry radios, which basically make their connection look like a serial port. In other words, you’ll have a pair of rf900 “telemetry radios”, one for the laptop your GCS is running on and one for the drone. Each radio will have a serial port or serial cable (or for the laptop, likely a serial-over-USB cable). The radios are paired, and their job is to make it look to the laptop and drone like they’re on either end of a simple serial cable.
One minor side note, on the ArduPilot Cube Orange+ (and possibly other Cubes), plugging in a single serial-over-USB cable connection will make two serial ports show up on your system, usually /dev/ttyACM0 and /dev/ttyACM1. You can run a mavlink session on each of these serial ports, without needing mavlink router or mavproxy.
This is for when you’re connecting to your drone via wifi or some other form of internet. The “other form of internet” is most likely a paired radio set, that work in a similar fashion to the serial port approach, i.e. you just plug into the ethernet port on the radio on either end and the radios make it look like they’re both plugged into the same switch.
There are at least four different common mavlink network connection types, but from everything I’ve read, the default, so-common-its-a-defacto-standard network type is: a) UDP based and also b) initiated from the drone to the pymavlink computer. In other words, the drone is configured to initiate the mavlink session by sending UDP packets to the GCS, and the GCS is configured to listen for those incoming UDP packets.
A typical UDP connection string looks like:
'udpin:0.0.0.0:5761'This indicates port 5761 on localhost. This is a bit overly complicated but still a very common configuration, the kind you would use for a pymavlink script running a companion computer that’s talking to mavlink-router or mavproxy running on the same companion computer (hence localhost) which is in turn talking to the flight controller over serial via a direct, physical serial cable and serial port.
If it’s a network connection, then it’s extremely likely that the network connection is being supplied by mavlink-router or mavproxy, running on the drone. See the section below, “Mavlink-Router and MAVProxy”.
In that case, the IP address and port you need for your connection string will be determined by either the endpoint in your mavlink-router config file, or by the parameters you use to start mavproxy. The mavlink-router config file is usually /etc/mavlink-router/main.conf, but you can customize that when you install mavlink-router.
Again, assuming that you’re following the defacto standard approach of UDP initiated from drone to GCS:
The drone starts up and starts sending UDP packets to the pymavlink computer’s IP address and UDP port.
Note, that means that the pymavlink computer’s IP address has to be configured the computer that’s running mavlink-router or mavproxy, which will be on the drone.
If you’re using mavlink-router, the pymavlink computer’s IP address will be in an endpoint defined in /etc/mavlink-router/main.conf.
If you’re using mavproxy, the pymavlink computer’s IP address will be in the parameters on the shell command used to start mavproxy.
When the pymavlink computer starts up it listens on the UDP port, receives the first UDP packets from mavlink-router or mavproxy
After the pymavlink script receives the first UDP packets from mavlink-router/mavproxy, it sends UDP packets back, doing some sort of handshake to set up the mavlink session.
Obviously there’s higher level mavlink logic for making the UDP traffic more session oriented. Normally I’d add a snarky comment here about “those who do not know TCP/IP are doomed to reinvent it”. But in this case there’s actually some valid justification, because we’re dealing with radio and the right thing to do with radio isn’t always the same as the right thing to do with an ethernet connection.
Specifically, the most obvious example is that with TCP/IP when you’re having trouble getting through, the problem is most likely network congestion, so you back off and try it more slowly. With radio, it’s most likely radio signal interference, and the best thing to do is to just hammer away, trying again and again.
Regardless, the “defacto standard”, such as it is, appears to be UDP initiated from the drone side to the pymavlink computer side. I emphasize this because, in my limited conversations with other programmers new to mavlink, they seem to find this highly counterintutive.
Mavlink-router or mavproxy are typically used to make the network version of mavlink happen in cases where you’re not using a straight up telemetry radio that pretends to be a serial port connection.
Note, however, that both mavlink router and mavproxy can also forward mavlink sessions via serial port (and also, see the section “Serial Port” for the note about the Cube Orange+ and two serial ports)
Typically the flight controller makes mavlink available over a serial port. Serial ports can only be controlled and accessed by one process at a time, so if you want more than one mavlink session at a time, the process that grabs the mavlink serial port has to provide mavlink to other processses.
Your options for that are primarily mavlink-router, but you can also use mavproxy to do that. Mavlink-router is generally considered “lighter weight” in terms of resource usage, so most peeople only use mavproxy if they want or need it for for other reasons anyway (like having a convenient command line GCS running on the flight controller hardware).
Mavlink-router is a compiled C/C++ app that multiplexes mavlink sessions and can be configured to provide mvalink over multiple serial ports or network ports.
MAVproxy is a python app (with some compiled C++ internals) that can also do this, but that’s not mavproxy’s intended job. Mavproxy is basically a command line GCS (Ground Control Station), but you can also give it start up parameters to provide mavlink on other serial ports or network ports, in much the same vein as mavlink-router.
Mavlink-router is lighter weight and just does the mavlink multiplexing and networking. It’s available from github, but only for Linux as far as I can find, not for Windows or Mac.
https://github.com/mavlink-router/mavlink-router
https://ardupilot.org/mavproxy/docs/getting_started/download_and_installation.html
Sometimes you want, or need, custom mavlink commands. This is a short summary of this topic.
Note that the receiving/processing end for all of these commands would be a Lua script running on ArduPilot. ArduPilot supports running lua scripts on the flight controller to customize behavior. Those lua scripts can register with ArduPilot to receive mavlink messages, and react to them by making other calls into ArduPilot.
Your options are:
From everything I’ve seen, #1 is just plain hacky (though “if it stupid but it works, it ain’t stupid”), #3 is a pain in the ass, #4 isn’t available in ArduPilot Stable yet. I lean towards #5, though #2 would work in a pinch.
Note: There are also the mavlink payload commands. These seem to be intended to be handled by custom implementations, so you could perhaps piggyback on them:
https://mavlink.io/en/services/payload.html
Two notes on logging in relation to this topic:
First, STATUSTEXT commands get logged on the drone (presumably in the dataflash logs). This is sort of analogous/opposite to the Lua function gcs_send_text(), which causes a message to be logged on the GCS.
Second, MAV_CMD_USER_1 messages get logged (“in MAVC”) according to ArduPilot contribuor Peter Barker.
Also, according to Peter Barker:
… and if you don’t need commands then DATA64 and friends also work.
This is really ArduPilot, not pymavlink, but as I point out above, you can’t really dig into pymavlink alone, it’s always going to be a combination of pymavlink, the mavlink protocol, and the mavlink implementations on ArduPilot (or whatever flight controller software you’re working with).
I might get back here later and add more detail, or I might make a different document. For now, here’s a very brief braindump.
In a nutshell, ArduPilot supports adding Lua scripts to your drone to extensively customize behavior. The Lua scripts can listen to mavlink commands coming into ArduPilot, send mavlink commands back to the GCS, make calls to ArduPilot to set values, etc.
Note that all lua scripts execute using the same resources, so if one lua script behaves badly, it can trip up other lua scripts.
To install a lua script on your drone, create an APM/scripts directory on your SD card and put your lua script in there. You can do this either by mounting the SD card on your workstation and directly editing it, or by inserting the SD into your flight controller, powering up the flight controller, connecting to the flight controller via Mission Planner or mavproxy, and then using Mission Planner/Config/MAVftp, or by using mavproxy MAVftp:
https://ardupilot.org/mavproxy/docs/modules/ftp.html
MANUAL> ftp path/to/local/myscript.lua APM/Scripts/myscript.lua
MANUAL> Putting path/to/local/myscript.lua as APM/Scripts/myscript.lua
Sent file of length 25213Note that, according to Peter Barker, unfortunately the mavftp implementation in mavproxy is really not suitable for use from your pymavlink script, so you can’t write a pymavlink script to automatically install your lua script.
Lua script errors will generally cause error messages to show up in the Mission Planner log or in the mavproxy session, though it doesn’t show up with any sort of prefix or indicator to make it clear that it’s a lua script error.
Note that in your mavproxy session, log messages from ArduPilot will have an “AP:” prefix. As I write this I don’t remember if Mission Planner’s logs have that “AP:” prefix.
In particular, a bad lua script error will stop the lua script from running, so the lua script error message will only show up at the beginning of the log, so it’s easy to miss.
You can use Mission Planner or the mavproxy scripting restart command to restart lua scripting, which makes it easier to see if the script runs or not.
Here’s an exampel of using scripting restart, and it also gives you an example of the “AP:” log messages.
MANUAL> scripting restart
Got COMMAND_ACK: SCRIPTING: ACCEPTED
AP: Scripting: stopped
Lua: State memory usage: 2796 + 16883
AP: Scripting: restarted
AP: starting up...
AP: Connecting to serial port...
AP: Starting up main_loop...
AP: 89,F8,1,24For now, this section is a grab bag of “useful links”. In other words, I’m sticking stuff here until I figure out a better place to put it. Generally speaking, this stuff should all be fairly tactical in scope, i.e. specific things, rather than broad topics.
This discussion thread has some really useful little tidbits. Read the whole thread.
In particular, buried in the code samples, he talks about mavutil.periodic_event(), which looks interesting. The class itself is simple, but the general design/approach for your main loop makes a lot of sense.
Note, periodic_event() itself isn’t that fancy, it’s just a bit of convenience code to manage tracking whether an event (e.g. sending a heartbeat, checking receive_match) is due to happen. However, the discussion/description in this thread, of writing the code this way, is interesting.
https://github.com/ardupilot/ardupilot/blob/master/Tools/autotest/arduplane.py
The ardupilot website has a bunch of python autotest scripts. To quote ArduPilot contributor Peter Barker:
In general, “how do I do X via mavlink” should be answered in the autotest suite. Big emphasis on the “should”, ’though.
https://mavlink.io/en/file_formats/
MAVLink systems often need to be able to store, exchange, or restore MAVLink information, including: mission plans, geofence definitions, rally points, parameters, logs, etc. Often the information is defined on one system and used on another (e.g. logs from an autopilot are parsed by analysis tools, missions are created using GCS planning tools and run from a companion computer).
There is a defacto standard used in many GCS systems and developer APIs for storing mission information: plain-text file format.
I wanted to understand pymavlink’s statefulness better, so I started tracing through the code. Specifically, every mavlink message sent has to have a SystemID, but you usually don’t set that explicitly in your pymavlink code. So where does that come from? Where do outoing messages get it?
I’m not done yet, in fact my first attempt, starting from mavutil.mavlink_connection() and connection.wait_heartbeat(), was a bit of a dead end. Still, here are my notes from tracing through that, it might be informative.
Note, since pretty much every implementation of mavlink starts by using the XML file that defines mavlink messages to generate C code and then bindings from whatever (in this case, python) to the binaries from that C code, it’s entirely possible that what I’m looking for happens inside the generated binaries.
https://github.com/ArduPilot/pymavlink/blob/master/mavutil.py
Source for the mavutil module, among many things it contains…
…the definition of the function mavlink_connection(), which you use to get your mavlink connection object and then you use the connection object to do everything else. This will in turn return an instance of one of several different connection classes:
Most of the time your connection object will be an instance of the mavudp() class. Using udp outbound from the flight controller to the GCS (or pymavlink script) for the mavlink connection seems to be the overwhelmingly common convention in the drone world, though I’d like to learn more about the tcp server mode (i.e. the flight controller listens for an inbound TCP connection.
https://github.com/ArduPilot/pymavlink/blob/5d0496a3ad654cd47f294b5baba027e25bfd15c7/mavutil.py#L557
This is the code that defines connection.wait_heartbeat(). Interestingly, contrary to what’s implied by various tutorials, there’s not really much in there. It just calls recv_match() to get the heartbeat:
def wait_heartbeat(self, blocking=True, timeout=None):
'''wait for a heartbeat so we know the target system IDs'''
return self.recv_match(type='HEARTBEAT', blocking=blocking, timeout=timeout)I mean, obviously, yeah, if you’re not getting heartbeats then you’re probably not connected and it’s pointless to send any commands.
And you can’t send any messages until you have the target system ID, which you get from the heartbeat message.
But you could easily write your own logic to manage that at a higher level.
First, mavlink_connection is defined here:
The mav_util.mavlink_connection() method looks at the parameters and then instantiates and returns the appropriate type of connection object, most commonly a mavudp object:
And mavudp is descended from mavfile:
https://github.com/ArduPilot/pymavlink/blob/9770f1d133779bb667e68cdf9565ee2ddd180c13/mavutil.py#L172
However, wait_heartbeat() just calls self.recv_match(type=‘HEARTBEAT’) and returns the message… so where does it grab the systemid?
https://github.com/ArduPilot/pymavlink/blob/9770f1d133779bb667e68cdf9565ee2ddd180c13/mavutil.py#L557
mavfile.recv_match() is here:
https://github.com/ArduPilot/pymavlink/blob/9770f1d133779bb667e68cdf9565ee2ddd180c13/mavutil.py#L507
Hm, nothing there, either.
Next step is to look at one of the send commands, trace that through and see where it picks up SystemID.
It’s quite possible, however, that what we’re looking for actually happens down in the generated C++ code (this is why I hate this sort of approach…).
To Be Continued.
https://ardupilot.org/dev/docs/mavlink-requesting-data.html
ArduPilot by default sends a variety messages back about the status of the system, etc, but it doesn’t send back all messages about all statuses. You can further customize this by using:
REQUEST_DATA_STREAM and SET_MESSAGE_INTERVAL to add messages to the list.
REQUEST_MESSAGE to request a single response.
This is a gotcha that ArduPilot contributor Peter Barker told me to watch out for. I’ve never run into it, but I’m including it here for reference.
SystemID and ComponentID are standard identifiers in mavlink messages and need to be unique for each mavlink-using-piece of your system. In other words, you need to make sure that the different parts of your system (the pymavlink script, mavproxy if you’re using it, each flight controller, etc) are not using the same values for these, or they’ll clash.
I’ve never explicitly set these values in my code, I’m not sure how pymavlink or mavproxy choose them, but I’ve also never had any known problems from conflicts in these values.
Peter Barker made the following suggestions to another user in the ArduPilot #pymavlink discord channel, I think they’re useful so I’m including a summary of them here, paraphrased to make them more general and more accessible.
Mavlink is a lossy protocol, you should expect that messages might be lost in either direction and plan for that.
In multi-link environments meaning communicating with multiple drones, you can get the same ACK twice, so your code should check for that.
When your code is talking to multiple vehicles, when you’re calling recv_match() to look for an ACK to a command, you should also check and confirm that the ACK is from the vehicle you sent the command to.
You should use a timeout on recv_match(), so it doesn’t just sit there, spinning and burning CPU time. Make your recv_match() call blocking, and add a 0.01 timeout parameter to the call.
When debugging with messages returned from recv_match(), call message.dump_message_verbose() and pass sys.stdout to dump_message_verbose() as the first parameter.
Consider accepting every message in the stream from the vehicle and make sure you’re not missing sequence numbers. [I think, here, he means check the sequence numbers on all the messages to detect if some messages did not get through.]
You already know what sort of message you’ve just received, you don’t need to to_dict it - just hit the attributes on the message directly.
Check not only for the ACK to your command but also that the command succeeded. For set mode commands, you could also base success off receiving a heartbeat with the correct mode in it, rather than a success ack.
If your structure allows for it, throw exceptions rather than returning False; assume the vehicle will do what you want and handle with exceptions where it doesn’t. You can then be more nuanced about your returned errors e.g. “not right now” vs “never”.
Consider running flake8 on your Python - your code-compile-test-swear loop can be shorter if you include running it through flake8 as part of that loop.
This page:
https://mavlink.io/en/mavgen_python/
says:
The [mavutil] link does not properly handle multiple systems running on the same port. If you need a multi-vehicle network see source-system-filtering.
And links to:
https://github.com/peterbarker/dronekit-python/tree/source-system-filtering/examples/multivehicle
Dronekit has poor support and this particular fork is 7 years old, but I’m including it because it looks like reading this (and perhaps looking at the source) might be informative about how mavlink and pymavlink handle SystemID, even if you don’t actually use this implementation.
You can use the mavlink message AUTOPILOT_VERSION to get the current version of ArduPilot installed on the controller
https://discord.com/channels/674039678562861068/930641827592306718/1245266227660591105
StephenDade The AUTOPILOT_VERSION message will give you what you want. See https://github.com/ArduPilot/MAVProxy/blob/master/MAVProxy/modules/mavproxy_useralerts.py#L65 for example UserA: I thought there was something about this one not existing in some versions of ArduPilot or not containing all the information? Looks like it doesn’t have the git hash, which can be rather critical… USerB: It’s fetching only “official-4.6.0”, it doesn’t return vehicle or frame type. StephenDade: Yeah, only applies to the more recent (<2 years?) versions of Ardupilot. Only gives the version numbers. Otherwise you need to start parsing the GCS messages StephenDade: Vehicle and frame type you can get from the heartbeat message
[5:46 PM]stevenjowens: @AndrewTridgell Question, the other day I was trying to figure out if there’s a simple way in mavproxy to “sniff” the mavlink messages, i.e. watch them going back and forth during a live session. The only way I found to do it was to use mavproxy logging and then use (or write) a tool/script to essentially tail the log, parse it and print it. Is there some other means?
In that particular case it was for somebody who was experimenting with writing their own simplified flight controller, for learning purposes, and trying to talk to their flight controller code with Mission Planner.
[11:22 PM]Peter Barker: watch *
[11:22 PM]Peter Barker: … or watch –verbose * if you’re want all the details
As an example, Peter Barker once told me that to set the autopilot version, first “watch AUTOPILOT_VERSION” and then “version”:
[12:05 PM]stevenjowens: The mavproxy command “fetch autopilot version” gets no response at all. https://github.com/ArduPilot/MAVProxy/blob/00afedf13050f49d8e87ccb0bb325313823edbf7/MAVProxy/modules/mavproxy_misc.py#L90 Is there a better or more appropriate command to get the version of the installed ArduPilot?
[8:38 PM]Peter Barker: It does get a response, but we don’t monitor the stream to print it. We could.
STABILIZE> STABILIZE> watch AUTOPILOT_VERSION STABILIZE> Watching [‘AUTOPILOT_VERSION’] version STABILIZE> Got COMMAND_ACK: REQUEST_MESSAGE: ACCEPTED < AUTOPILOT_VERSION {capabilities : 64495, flight_sw_version : 67502080, middleware_sw_version : 0, os_sw_version : 0, board_version : 0, flight_custom_version : [48, 98, 56, 54, 54, 97, 52, 98], middleware_custom_version : [0, 0, 0, 0, 0, 0, 0, 0], os_custom_version : [0, 0, 0, 0, 0, 0, 0, 0], vendor_id : 0, product_id : 0, uid : 0, uid2 : [52, 48, 51, 51, 56, 99, 49, 50, 49, 101, 99, 100, 52, 53, 98, 48, 98, 0]}
This is Mavproxy, but it looks potentially useful for development and debugging, so I’m pasting it in here.
https://discuss.bluerobotics.com/t/how-use-pymavlink/5674/7
Patrick José Pereira patrickelectricBlue Robotics Engineer Aug 2019
There is a new feature that creates a REST API in mavproxy 43 allowing an easy access to any mavlink information.
https://github.com/bluerobotics/companion/issues/264
You can check the status of such feature in companion here:
https://github.com/ArduPilot/MAVProxy/pull/610
[…] This feature is available since mavproxy 1.8.7 (from May), to use it in companion, you need to upgrade the installed mavproxy version, install the necessary packages for the SERVER side and to configure mavproxy to provide the REST API server in the desired port that you wish to use.
[…] Willian Galvani Blue Robotics Engineer Aug 2019 y Hello Justin,
You can install mavproxy and the optional requirements for the rest server with pip install mavproxy[server] but I think you will run into some dependency issues (It’s been a while, I don’t remember quite well).
You can then set your mavproxy to something like this:
–master=/dev/autopilot,115200 –load-module=‘GPSInput,DepthOutput,restserver’ –source-system=200 –cmd=“set heartbeat 0;restserver start;restserver address 0.0.0.0:5001;” –out udpin:localhost:9000 –out udpbcast:192.168.2.255:14550 –mav20 –aircraft telemetry –streamrate -1
https://github.com/ArduPilot/MAVProxy/pull/610
patrickelectric Member patrickelectric commented Apr 16, 2019 •
Add rest server: module load restserver restserver start restserver stop restserver address 127.0.0.1:3333
Allow information access via simple rest api:
localhost:5000/rest/mavlink/VFR_HUD/
The official pymavlink page made a lot more sense the third or fourth time around, after I’d gleaned enough of the big picture from reading various scattered docs:
https://mavlink.io/en/mavgen_python/
Since the official pymavlink docs don’t include an API doc (but see the samba.org link, below) check the mavlink.io docs for the mavlink messages. They’re sometimes a bit ambiguous about the message field data types, etc, but still, the spec is the authoritative source:
https://mavlink.io/en/messages/common.html
The mavlink microservices are documented here:
https://mavlink.io/en/services/
The ArduPilot project also has a bunch of pymavlink examples on their github repo. They’re useful, especially once you get a sense of the bigger picture:
The Google Group for mavlink: https://groups.google.com/g/mavlink
https://github.com/ArduPilot/pymavlink/tree/master/examples
The ArduSub project has some of the better examples of pymavlink code that I’ve seen:
https://www.ardusub.com/developers/pymavlink.html
The Intelligent Quads youtube channel has some decent videos on pymavlink:
https://www.youtube.com/playlist?list=PLy9nLDKxDN68cwdt5EznyAul6R8mUSNou
Note, all example code is at:
https://github.com/Intelligent-Quads/iq_pymavlink_tutorial
For pymavlink API docs, Andrew Tridgell (original author of pymavlink and mavproxy, and still active in the ArduPilot project) at one point suggested (but I haven’t tried it yet):
Normally I just use pydoc3 eg. pydoc3 pymavlink.mavutil.mavlink_connection
I’m still not sure of the backstory on this page at samba.org, but these are some somewhat useful API docs for pymavlink:
https://www.samba.org/tridge/UAV/pymavlink/apidocs/
This is one of the better overall articles on pymavlink that I found:
[https://medium.com/@tonyjacob_/using-mavlink-for-custom-applications-466e1d632f84](https://medium.com/@tonyjacob_/using-mavlink-for-custom-applications-466e1d632f84)
Tony Jacob above in turn links to this paper, which is a good background doc on the MAVLink protocol:
This is just a gist but it gives a fairly comprehensive example of writing a pymavlink app using python’s asyncio:
https://gist.github.com/patrickelectric/d6264fe309d01b44d5540d75cc45cb83
This is for the currently-languishing dronekit, but it looks like they have very good docs and are worth a read.
https://dronekit-python.readthedocs.io/en/latest/guide/copter/guided_mode.html
The ArduPilot “discuss” forums have a lot of useful information on pymavlink as well as on ArduPilot:
Several of the links in this document are to useful posts on diydrones.com.
https://diydrones.com/main/search/search?q=pymavlink
This is a fairly interesting article:
https://vwangsf.medium.com/the-state-of-mavlink-in-2021-5f33c4fbde92
Besides pymavlink there’s also Dronekit, MAVSDK, and MAVROS.
All of these are higher level than pymavlink (Dronekit is built on top of pymavlink), but they all have their own problems and drawbacks. I’m not interested in arguing them – I have no experience with them – but I’ll mention them here, to get you started on looking into them.
Dronekit is built around pymavlink and is higher level, in the sense that you ask Dronekit to do things at a higher logical levle. Dronekit seems nice, but it also seems to be languishing without any real support. From its github it’s been literally five or more years since its last update, and the github main readme for dronekit is seeking volunteers for maintaining it.
Peter Barker of the ArduPilot open source project is the official maintainer of dronekit, but I think that’s just because somebody has to be. If you’re interested in helping, please volunteer!
https://github.com/peterbarker
MAVSDK is also higher level. MAVSDK is primarily written in C++ and has bindings for a number of different languages, including python.
MAVSDK is developed primarily for PX4. When I started down this path, ArduPilot was explicitly not supported by MAVSDK. Since I’m more interested in ArduPilot that’s an issue for me, so I didn’t look further into it. Since then, supposedly ArduPilot is supported, but I’ve heard vague things about problems with it. I don’t know if those are due to the PX4/ArduPilot mismatch, or something else.
Julian Oes is the author of and maintainer of MAVSDK:
https://discuss.px4.io/u/JulianOes/summary
MAVROS is the mavlink library for ROS, the Robot Operating System. I hear good things about MAVROS from people who like ROS. I hear bad things about ROS from people who don’t like ROS.
Generally the attitude in the drone programming community seems to be, if you’re using ROS already, use MAVROS, otherwise MAVROS isn’t worth the extra hassles of using ROS. Your mileage may vary.