System-testing embedded code in Rust, part one: Infrastructure

Previously on #rust:

• Rust crate release checklist
• Things I learned writing an SSDP implementation in Rust
• Self-hosted CI for Rust and C++ using Laminar

One of the goals of the Rust crates I’ve been working on, cotton-netif and cotton-ssdp, is for them to be useful on embedded systems: microcontroller-based devices only capable of running simple real-time operating systems, as opposed to full-size Linux systems.

There is good support in Rust itself for targetting such platforms: it’s relatively easy to write such code (with the no_std attribute), and not even much harder to cross-compile it (using Cargo) and even run it on the target (the probe-rs folks do great work there). But if you’ve read some of the other posts here, you’ll be familiar with the idea that software isn’t done until it can be repeatably shown to be done. Someone — perhaps not Themis, the goddess of Justice, as pictured; more likely Laminar, the goddess of CI — must solemnly, dispassionately, objectively weigh the code’s activities against (some representation of) its specification, and hold it in judgement if it falls short.

Less fancifully, what’s needed is an automated way for CI (running on a rich, non-embedded host platform) to run the code on a genuine, embedded, target platform and check its functionality. Of course, it’s best to arrange that as as much code as possible is abstracted away from the hardware so that it can be unit-tested on the host, run through Miri and Valgrind and other dynamic-analysis tools on the host, and just plain debugged on the host, where everything is a little less awkward. But the proof of the pudding is still in the eating: only tests that run on the target can be the final arbiter.

(In particular, whether the target hardware appears well-documented or not, it’s all too easy to have misconceptions about how peripherals behave, leading to a situation where the code and the host-side unit-test agree about what’s going on, but they’re both wrong because the actual hardware does something completely different.)

At Electric Imp we had (what became) a large subsystem of Python scripts which our CI system ran, and which in turn installed the newly-built device firmware on some dozens of Imp devices, running through system-tests including thorough regression-testing of the wifi connectivity, the “curated” peripherals, and really the entire of the firmware functionality. This worked extremely well, and over the years caught simply oodles of bugs which had passed unit-testing but would have in some way scuppered our customers’ actual devices — but all of that Python always felt like an add-on to the main C++ build system, needing different skills to maintain. As Rust (or at least Cargo), by contrast, quite rightly represents testing as a first-class language feature, I wondered whether Cargo’s own facilities could be used to system-test embedded Rust without having to invent lots of extra infrastructure bolted on the side.

In this blog post I’ll focus on getting the testing infrastructure set up, literally just far enough to get a target-side test that prints “Hello, World” and a host-side test runner which checks that it has done so. Adding actual tests for the crates’ functionality (SSDP, to start with) will likely come in a subsequent post; CI considerations in yet another.

Goals of the Cotton automated system-tests

1. Joel Test #2 compatibility — Joel Spolsky, writing some years ago now, has some pithy questions to ask software development teams. “Can you make a build in one step?” is always valid to ask, for the reasons he lists — mostly about not forgetting intermediate steps — but also because in practice what’s easy is what most people will do. If the easy thing to do is run a command that builds absolutely everything, then most of the time developers will run that command. (Except perhaps if it starts to take too long, but that’s a different issue.) And the more that’s built by the one command that everybody uses each day, the harder it is to accidentally introduce a bug that affects some builds or facets of the system but not others.

2. Easy for me/anyone/CI to test everything on the host — Because most of the code compiles for the host (potentially even parts that are only useful on the target), host-side development should remain straightforward. In particular, it mustn’t require the presence of a cross-toolchain, or nightly Rust, or any target hardware.

   It certainly mustn’t require people using the crates from crates.io in normal host-side builds in the normal way, to install or attach anything special. (But Cargo makes sure of that anyway.)

   Someone wanting to work on a Cotton crate in combination with their own project, can check out the Cotton repository alongside their own, and use a path dependency when importing the Cotton crate (just as they would for a one-crate upstream repository).

3. Easy for me/anyone/CI to test everything on one target — Even if Cotton eventually targets many different embedded systems, almost all embedded developers will only have one type of hardware attached to their development host at any one time. It must be straightforward to run all the tests that can run on (say) an STM32F746-Nucleo, but none that requires different hardware.

4. Easy for me/anyone/CI to test one crate on one target — Eventually several crates (not just SSDP) will share the same system-test infrastructure; it must remain possible to test just one crate at a time, for the sake of development cycle time.

5. Possible for CI to test everything on many/all targets — Once Cotton targets several different embedded systems, each one should have its tests run by CI. But this mustn’t require one CI host per target — it must be possible to attach several development boards to the same CI host and have it run the right tests on the right boards. Notice that this goal is for it to be “possible”, not “easy”: having lots of different development boards attached is going to be uncommon, so it’s okay for it to need slightly more awkward setup.
6. Separation of concerns — The cotton-ssdp crate, say, can probably be tested on each of several different target devices. Adding a new target device mustn’t require changes to cotton-ssdp itself.

Outline of the solution

You can see the merge that creates this infrastructure at commit 181a8fdc and the whole tree at that commit here on Github.

There were definitely a few false starts along the way to achieving those goals. The first thing I tried was “per-package targets”; this is a Rust facility that in theory should make it possible to mark certain packages in a workspace as building for a different platform than the rest of the workspace. The inspiration for the feature was people building web apps where the server end compiles to x86_64 or ARM64 or whatever’s cheap in AWS these days, but the client end compiles to WASM to run in browsers. It’s more-or-less exactly what’s needed here too — but sadly it’s more complicated to implement than you’d first think, and is only available in nightly Rust where it doesn’t work very well. (When I tried it, cargo test kept trying to run my STM32 binaries on the host.)

Without per-package targets, I was going to need different invocations of Cargo for host and device builds (because each invocation of Cargo can only build for a single platform). So I tried having a build.rs build script that, when invoked for the host platform, re-runs Cargo for the device platform. Yee-hah, right? Sheer cowboyery. It doesn’t work, because Cargo deadlocks trying to rebuild the same crate it’s already building. Can’t really blame it, either.

I looked for a while into setting rustflags in Cargo.toml, but that can’t be set per-package in a single workspace, let alone per-test in a single package. Each different set of rustflags must currently be a separate invocation of Cargo.

The answer seems to be, to take parts of each of those ideas:

• Have the workspace as a whole continue to build native for the host,
• and have a build script that re-invokes Cargo for each target platform,
• but have each target platform’s root crate be separate, and never built for the host, using an exclusion in the workspace Cargo.toml:

  Cargo.toml
  [workspace] members = [ "cotton-netif", "cotton-ssdp", "systemtests", ] exclude = [ "cross", ]

The resulting workspace structure looks like this:

cotton ├── Cargo.toml ├── cotton-netif │   ├── Cargo.toml │   └── ... ├── cotton-ssdp │   ├── Cargo.toml │   └── ... ├── cross │   └── stm32f746-nucleo │       ├── .cargo │       │   └── config.toml │       ├── Cargo.toml │       ├── memory.x │       └── src │           └── bin │               └── hello.rs └── systemtests     ├── build.rs     ├── Cargo.toml     ├── src     │   └── lib.rs     └── tests         └── stm32f746-nucleo.rs

The cross subdirectory is excluded from the root workspace (in the root Cargo.toml), and the crates inside it (at present, only stm32f746-nucleo) are built by a recursive Cargo invocation in systemtests/build.rs; recursively invoking Cargo is a bit subtle, as you need to unset a bunch of environment variables in order that the sub-Cargo runs mostly as a new top-level Cargo (otherwise, the deadlocking issues reappear). Here’s the build-script section that achieves that, with the cross-compilation guarded by a (Cargo) feature called arm:

systemtests/build.rs (partial)

if env::var("CARGO_FEATURE_ARM").is_ok() { /* Run the inner Cargo without any Cargo-related environment variables * from this outer Cargo. */ let filtered_env: HashMap<String, String> = env::vars() .filter(|(k, _)| !k.starts_with("CARGO")) .collect(); let child = Command::new("cargo") .arg("build") .arg("-vv") .arg("--all-targets") .arg("--target") .arg("thumbv7em-none-eabi") .current_dir("../cross/stm32f746-nucleo") .env_clear() .envs(&filtered_env) .output() .expect("failed to cross-compile for ARM"); io::stdout().write_all(&child.stderr).unwrap(); io::stdout().write_all(&child.stdout).unwrap(); assert!(child.status.success()); }

The obvious downside here, is that build scripts aren’t run with a live terminal: unless the outer Cargo is invoked with -vv, the script’s standard output and standard error are written only to files, and shown only if the script fails. If the recursive Cargo invocation succeeds, all you see is a long pause in your build — though at least if the recursive invocation fails, you do see its error output.

For the time being, this acts as extra encouragement to keep complex code out of the only-compiled-for-target crates — though if need be you can always do a normal top-level Cargo build inside the STM32 crate, to see live standard output and standard error:

cargo -C cross/stm32f746-nucleo build --target thumbv7em-none-eabi

Host-side runner for a device-side test

To start with (and to validate the rest of the solution), there is only one actual system-test in the first merge: writing the “Hello World” binary onto an STM32F746-Nucleo development board, running it, and checking the output. Cargo will have run our build script before running the test, so we know that our device-side binaries have all been built and are up-to-date (another Joel Test benefit) — and it’s just a question of using (the excellent) probe-run, which is built on probe-rs and already knows about STM32 development boards, to write the binary to the STM32 chip and then run it. The STM32 binary uses defmt-rtt for its logging output, using the Cortex-M Real-Time Tracing system, support for which is again built-in to probe-run (no semihosting! no UARTs!), so the device side is as simple as this:

cross/stm32f746-nucleo/src/bin/hello.rs
#![no_std] #![no_main]   use defmt_rtt as _; // global logger use panic_probe as _; use cortex_m::asm;   #[cortex_m_rt::entry] fn main() -> ! { defmt::println!("Hello STM32F746 Nucleo!");   loop { asm::bkpt() } }

and the host side only a little less simple:

systemtests/tests/stm32f746-nucleo.rs

use assertables::*; use serial_test::*; use std::env; use std::path::Path; use std::process::Command;   use std::io::{self, Write};   #[test] #[serial] #[cfg_attr(miri, ignore)] fn arm_stm32f7_hello() { let elf = Path::new(env!("CARGO_MANIFEST_DIR")).join( "../cross/stm32f746-nucleo/target/thumbv7em-none-eabi/debug/hello", );   let mut cmd = Command::new("probe-run"); if let Ok(serial) = env::var("COTTON_PROBE_STM32F746_NUCLEO") { cmd.arg("--probe"); cmd.arg(serial); } let output = cmd .arg("--chip") .arg("STM32F746ZGTx") .arg(elf) .output() .expect("failed to execute probe-run");   println!("manifest: {}", env!("CARGO_MANIFEST_DIR")); println!("status: {}", output.status); io::stdout().write_all(&output.stderr).unwrap(); io::stdout().write_all(&output.stdout).unwrap(); assert!(output.status.success());   let stdout = String::from_utf8_lossy(&output.stdout); assert_contains!(stdout, "Hello STM32F746 Nucleo"); }

Notice the use of serial_test to defeat Rust’s default behaviour of running multiple integration tests in parallel — that wouldn’t end well if they were all competing for one single physical development board. (Although obviously there’s only one test at the moment.)

Clearly much more complex logging and checking could (and will) be implemented via the same mechanism, but as a proof-of-concept this suffices for this initial blog post.

Does this meet our goals?

1. ✅ Joel Test — the commands:

   cargo build
   cargo test
   cargo build-all-features --all-targets
   cargo test-all-features --all-targets
   

   all do as expected, and none need any cross-toolchains, special hardware, or even nightly Rust. (The all-features ones work because the arm feature is carefully excluded from all-features builds.) The same generic “CI for Rust” scripts that worked for simple old host-side Cotton, continue to work fine with the exciting new multi-platform Cotton.
2. ✅ Test everything on the host — as above, the easy, everyday commands build and test all the host-compatible crates. The device-only crates can be built with:

   cargo build -F arm
   cargo test -F arm
   

   This needs the thumbv7em-none-eabi target to be installed for the current Rust toolchain:

   rustup target add thumbv7em-none-eabi

   but does not need any hardware, nor nightly Rust.
3. ✅ Test everything on one target — The Cargo.toml in the systemtests declares a Cargo feature stm32f746-nucleo, which depends on feature arm and enables the integration test whose host side is shown above. So building and running this test (i.e., running all the tests for that particular development board) looks like this:

   cargo build -F arm,stm32f746-nucleo
   cargo test -F arm,stm32f746-nucleo
   

   This needs the cross-toolchain installed as above, and of course it needs an actual physical STM32F746-Nucleo development board attached via USB to the host computer running the tests.
4. ✅ Test one crate on one target — Because the system-tests infrastructure is shared, this would have to be accomplished by careful naming of tests, and the use of Cargo’s wildcard test name option:

   cargo test -F arm,stm32f746-nucleo --test '*ssdp*'
   

5. ✅ Test everything on many targets — the main issue here is that, if several different (probe-rs compatible) development boards are attached to the same host, the probe-run command needs to be told each time which one to use. This is the idea behind the optional COTTON_PROBE_STM32F746_NUCLEO environment variable seen in the host-side runner above: a CI or other setup that has several development boards attached, needs to specify using these environment variables the unique “probe identifier” of each one. This feature will get more of a workout in a future blog post when a second target platform is added.

6. ✅ Separation of concerns — So far, this is good but not perfect. Device-side code that can be compiled for the host goes in the top-level workspace (perhaps alongside host-side code that can’t be compiled for the device). Device-side code that can’t be compiled for the host, goes in one of the crates under the “cross” directory. (Maybe one day those too should become workspaces?) Entire device-side applications (or, at the very least, example ones) can live there too.

   At some level it would be appealing for (say) system-tests for SSDP to go somewhere under the cotton-ssdp create. But device-side tests are inherently device-specific, and given N crates and M devices and a potentially N×M-sized testing matrix, it seemed better to keep the tests by device rather than by crate-under-test, on the basis that all engineers working with embedded Rust (in the device directories) are surely also familiar with host-side Rust, but the reverse (engineers working on host-side Rust in, say, the SSDP crate being familiar with embedded Rust) seems less guaranteed.

   Also, all the code in the systemtests crate, including much of build.rs, is generic across any crate wanting system-testing, and isn’t Cotton-specific. As of this blog post, no better advice is being offered here than to copy-and-paste it into your own projects, but in the future it would be more Rust-like to offer this functionality in free-standing crates that other projects desiring system-testing could just import in the normal way.

That Github link once again

You can see the merge that creates this infrastructure at commit 181a8fdc and the whole tree at that commit here on Github.

Similar blogs elsewhere

Ferrous Systems have a series of blog posts covering very similar topics, though they use cargo-xtask to construct their single build commands.

Continue to...

• System-testing embedded code in Rust 2: Testing SSDP
• System-testing embedded code in Rust 3: A CI runner

Perceived angular bisection

Given collinear points A, B, C in the plane, the locus of points P such that $\angle \mathrm{APB}=\angle \mathrm{BPC}\ne 0$ is the unique generalised circle through B which inverts A to C.

Those angles are also trivially equal (both zero) for all P collinear with A, B, C and not between A and C. The locus of other points P is a true circle, except when $AB\; =\; BC$, in which case it is the straight line that is the perpendicular bisector of AC through B; "generalised" circle includes this case as a notional "circle of zero curvature".

Equivalently, the locus is the circle through B centred at a point O chosen such that $\mathrm{OA}.\mathrm{OC}={\mathrm{OB}}^2$ (again with the exception when $AB\; =\; BC$).

This result answers the question: given three points in a line and no other information — three lamp-posts in an otherwise-dark landscape, for instance — whereabouts are the viewpoints from which the middle one appears, through perspective, to be exactly halfway in-between the other two?

Proof

Consider such a point P. Without loss of generality, assume AB < BC (the AB=BC case is trivial, and the AB > BC case is symmetrical with this one).

Draw the unique circle through P and B whose centre is collinear with A, B, and C (e.g. by constructing the perpendicular bisector of PB and marking its centre where that line intersects with the line through A and C). Call this centre O.

Label the angles as shown: $$\begin{array}{cccc}\angle \mathrm{OPA}& =& c& \\ \angle \mathrm{APB}& =& d& \text{which is also}\angle BPC\\ \angle \mathrm{OCP}& =& e& \\ \angle \mathrm{OBP}& =& g& \end{array}$$

Then consider:

$$\begin{array}{cccc}\text{in △CPB,}& e+d+(\mathrm{180°}-g)& =& \mathrm{180°}\\ & e+d& =& g\\ \text{△POB is isosceles,}& c+d& =& g\\ \text{therefore,}& e& =& c\\ \text{i.e.,}& \angle \mathrm{OPA}& =& \angle \mathrm{OCP}\end{array}$$

But the triangles OAP and OPC also have the angle at O in common, so they have two angles in common, therefore they are similar, therefore the ratios of corresponding sides are equal: $\mathrm{OA}/\mathrm{OP}=\mathrm{OP}/\mathrm{OC}$.

And $\mathrm{OP}=\mathrm{OB}$, because both are radii of the circle, so $\mathrm{OA}/\mathrm{OB}=\mathrm{OB}/\mathrm{OC}$ or $\mathrm{OA}.\mathrm{OC}={\mathrm{OB}}^2$, which is one definition of the circle of inversion.

Note that the position of O is the same whichever P we pick; it doesn't depend on any of the angles. (This is easier to see if we rearrange $\mathrm{OA}.\mathrm{OC}={\mathrm{OB}}^2$ into $\mathrm{OA}=\frac{{\mathrm{AB}}^2}{\mathrm{BC}-\mathrm{AB}}$ — it's uniquely specified by the positions of A, B, and C.) So the circle does represent the locus of all P.

Bit off-topic for this blog though eh

Yes. But it is a problem I've been thinking about for a while — in fact, ever since I read that, given the original formulation (three lamp-posts on a darkened plain) you can't even tell from a photograph whether or not they're evenly-spaced, because perspective means that they could be anywhere. But if your photograph has four lamp-posts in, you can tell whether or not they're evenly-spaced, because the cross-ratio is projection-invariant.

So I wondered whether, given three lamp-posts, there was always a point from which they appear evenly-spaced. It felt sort-of plausible that there were such points, but I had no intuition what their locus looked like. The first time I attacked this problem I plotted the points by brute force, and was surprised to see an apparently perfect circle. With the cosine rule and lots of ghastly slog (and Wolfram Alpha) I found the equation of the circle and the position of O. But that was clearly not The Book's proof, so I shelved the draft blog post — for some years. Then I happened to be reading about circle inversion, and suddenly realised it was talking about the three-lamp-posts circle. Eventually I was able to use circle-inversion techniques to come up with a geometrical, not algebraic proof, which I'm much happier with.

And if you're dying to see how bad the ghastly brute-force proof looked, here's a part of it ($p$ is $\mathrm{AB}$ and $q$ is $\mathrm{BC}$):

$p^2{q}^2{x}^2-2p^{2}q{x}^3+p^2{x}^4+p^2{x}^2{y}^2+2p{q}^2{x}^3+2p{q}^{2}x{y}^2-4pq{x}^4-4pq{x}^2{y}^2+2p{x}^5+4p{x}^3{y}^2+2px{y}^4+q^2{x}^4+2q^2{x}^2{y}^2+q^2{y}^4-2q{x}^5-4q{x}^3{y}^2-2qx{y}^4+x^6+3x^4{y}^2+3x^2{y}^4+y^6=p^2{q}^2{x}^2-2p^{2}q{x}^3-2p^{2}qx{y}^2+p^2{x}^4+2p^2{x}^2{y}^2+p^2{y}^4+2p{q}^2{x}^3-4pq{x}^4-4pq{x}^2{y}^2+2p{x}^5+4p{x}^3{y}^2+2px{y}^4+q^2{x}^4+q^2{x}^2{y}^2-2q{x}^5-4q{x}^3{y}^2-2qx{y}^4+x^6+3x^4{y}^2+3x^2{y}^4+y^6$

(I didn't set that by hand; there used to be an online Latex-to-MathML converter, but it seems to have since been retired.)

Rust crate release checklist

Previously on #rust:

• Things I learned writing an SSDP implementation in Rust
• Self-hosted CI for Rust and C++ using Laminar

There’s been a few public releases now of the Rust crates I’ve been working on, cotton-netif and cotton-ssdp. The SSDP one even has a merged pull request from a contributor! But because it’s often a little while between releases, I struggle to remember all the steps required. (Not so many, as Cotton is no massive failer of the Joel Test, but there are a few moving parts just because of all the fine, free open-source tools in use.) This document collects them all in one place, as much for my own benefit as anyone else’s.

1. Check Github for third-party pull requests as I don’t want to annoy contributors by seeming to ignore their work. This check can be automated using a badge:
2. Update Cargo.toml and CHANGELOG.md for all packages being released, like in commit 4ba3675c. If the package being released is depended on by other parts of Cotton, update their Cargo.toml dependencies too, like in commit c989d9f1.
3. Check that everything is pushed upstream, both to self-hosted CI and to Github.

   git push main git push github main

4. Check that both CI pipelines are passing; again, there’s a badge for the Github one:

   If your CI includes statistical metrics (as opposed to pass/fail ones: coverage, for example), check that those are in acceptable ranges too.

5. Do a dry-run publish, remembering to cd to the crate directory, not the workspace root:

   cargo publish --dry-run

   Cargo will check that the package is buildable; if any errors occur, fix them and go back to Step 3.
6. Tag the release, using multiple tags if multiple crates are being released:

   git tag cotton-ssdp-0.0.3

7. Push the new tag to both upstreams:

   git push cotton-ssdp-0.0.3 git push github cotton-ssdp-0.0.3

8. Actually publish the crate on crates.io:

   cargo publish

   Hopefully there won’t be any errors, given that the dry-run succeeded.
9. Let any contributors know that their stuff is now in a release – if any pull requests have been merged, now is the time to let those contributors know that they can go back to using real upstream releases of your crate, and potentially stop maintaining their forks.

Three SSH settings which aren’t the default, but which you probably want

Previously on #homelab:

• Setting up a Raspberry Pi as home server
• Hub-and-spoke backups using Syncthing
• Self-hosted CI for Rust and C++ using Laminar
• Network firewall on a Raspberry Pi using OpenWRT

Everybody uses OpenSSH to securely log in to remote machines. And they have done for ages. But that’s actually a problem, because OpenSSH has been around for so long that some of the security decisions made earlier in the project’s history no longer match current best practices. Here are a few things which would probably be the default if OpenSSH was starting out today, but which – for sound backward-compatibility reasons – you’ll need to arrange for yourself.

“ssh-add -c”

One criticism of the ssh-agent system is that when using it, you lose visibility of exactly when you’re signing for things. One way to mitigate this is, when using ssh-add to add local private keys to the agent, adding the -c option. This makes ssh-agent ask for confirmation (on console or in a pop-up dialog) every time a private-key operation is requested; unexpected pop-ups could be a sign that nefarious software is trying to use your key.

This also somewhat mitigates the risk, when using the agent-forwarding (ssh -A) feature, of attacks by a malicious actor who has root on the remote computer.

“ssh-add -D” on screen lock

On most reasonable systems, you have to give your local Unix password to unlock the screen once the screensaver has kicked-in. But if you feel that your SSH private key is more valuable than your local password – which you probably do, otherwise you wouldn’t’ve bothered encrypting your SSH private key in the first place – then that’s effectively a privilege-escalation attack: if you’ve left a ssh-agent session running, then knowing only your local password gives an attacker login ability using your SSH private key.

The obvious way to mitigate that, is to empty all saved keys from the ssh-agent session, every time the screen locks. Honestly it’s a bit surprising that KDE and Gnome don’t already have this built-in – but (at least in KDE) there’s a hook that lets you do just that.

In KDE “System Settings”, go to “Notifications” then next to “Applications:” click “Configure...”. Scroll down to “Screen Saver” and click “Configure Events...”. In the resulting pop-up window, choose “Screen locked” then below tick “Run command” and enter:

/usr/bin/ssh-add -D

Alternatively, manually add the following to ~/.config/ksmserver.notify:

[Event/locked] Action=Execute Execute=/usr/bin/ssh-add -D Logfile= Sound= TTS=

Under Gnome it’s less straightforward; there’s a script available called lockheed, but as it stands it only listens for unlock events; perhaps it would be possible to modify it to listen for lock events too. (Or perhaps wiping the keys from ssh-agent only on unlock is actually okay.)

“ssh-keygen -a 1000”

Over the years, OpenSSH has gone through a few design iterations on how to encrypt SSH private keys – in other words, what the passphrase you give to unlock the key actually does. If your private key file (usually ~/.ssh/id_rsa) starts with

-----BEGIN RSA PRIVATE KEY----- Proc-Type: 4,ENCRYPTED ...

then it is encrypted in a very obsolete way using PKCS#1; such passphrases are vulnerable to brute-forcing, should an attacker get their hands on the file. If, alternatively, it starts with:

-----BEGIN ENCRYPTED PRIVATE KEY----- MII...

then it’s PKCS#8, which is better but still not great. What you hope to see is the OpenSSH key format, which allows specifying the number of rounds of key-derivation function to use (i.e., how much work an attacker would have to do per guess in order to brute-force the passphrase):

-----BEGIN OPENSSH PRIVATE KEY-----

This particular recommendation slightly undermines the title of the blog post, because the OpenSSH format, which you probably want, is the default when creating new keys nowadays – but only since the OpenSSH 7.8 release of 2018-Aug-24, and OpenSSH does not upgrade from one format to another automatically. (Which is the Right Answer of course, as doing so would break compatibility with older versions if an upgrade ever had to be rolled-back.) This means that if your key was created with an older version of OpenSSH, the encryption used was likely to be, and is likely to still remain, one of the weaker forms.

Fortunately, there’s an OpenSSH facility to update just the encryption of any private key to the newest, most secure format, without altering the actual key:

ssh-keygen -p -f id_rsa -a 1000

This command upgrades the passphrase protection. It asks for the old passphrase (to decrypt the key) and then twice for the new passphrase (to re-encrypt it). If you’re confident that your old private key hasn’t leaked anywhere, you can re-use the same passphrase. (If you aren’t confident of that, you probably need to generate all-new keys anyway.) The -a 1000 sets the number of rounds to 1,000 – up from the default of 16. On this oldish Core-i7 machine, the setting of 1,000 makes checking the passphrase take about ten seconds. (Whether successful or unsuccessful!) This is a slight annoyance for you, but each time you’re waiting those ten seconds you can be thinking about how any attacker trying to brute-force your passphrase will be using up all that CPU on every single wrong guess they make.

Unfortunately, although the number of rounds is stored unencrypted in the key file, there appears to be no straightforward way of reading it out again directly. Following the directions given in a stackoverflow answer, you can use this command to get a hex dump of the base64-decoded encrypted key structure:

cat id_rsa | head -n -1 | tail -n +2 | base64 -d | hexdump -C | head

On a key I had lying around, this produced the bytes shown below. What you’re looking for is the string “bcrypt”, followed by 24 bytes you don’t care about (the KDF-descriptor length 0x0000_0018, the salt length 0x0000_0010, and the salt itself as 16 random bytes) followed by a 32-bit big-endian value which is the number of rounds:

00000000 6f 70 65 6e 73 73 68 2d 6b 65 79 2d 76 31 00 00 |openssh-key-v1..| 00000010 00 00 0a 61 65 73 32 35 36 2d 63 74 72 00 00 00 |...aes256-ctr...| 00000020 06 62 63 72 79 70 74 00 00 00 18 00 00 00 10 1c |.bcrypt.........| 00000030 d1 ab a0 6b cd 50 a7 8e 01 8c 9a f7 98 32 a6 00 |...k.P.......2..| 00000040 00 00 10 00 00 00 01 00 00 00 33 00 00 00 0b 73 |..........3....s|

In this file it’s 16 (0x0000_0010), the default. Once I’d run the ssh-keygen command on it, it instead appears as 1,000 (0x0000_03e8):

00000000 6f 70 65 6e 73 73 68 2d 6b 65 79 2d 76 31 00 00 |openssh-key-v1..| 00000010 00 00 0a 61 65 73 32 35 36 2d 63 74 72 00 00 00 |...aes256-ctr...| 00000020 06 62 63 72 79 70 74 00 00 00 18 00 00 00 10 cb |.bcrypt.........| 00000030 d4 4a be 47 b2 26 c9 15 d4 8d 0d d0 36 4c 62 00 |.J.G.&......6Lb.| 00000040 00 03 e8 00 00 00 01 00 00 00 33 00 00 00 0b 73 |..........3....s|

Whenever you need to generate a new SSH key, the ssh-keygen command accepts the -a 1000 option in that case too.

Bonus fourth SSH thing while you’re here

The encfs-agent script lets you set up EncFS encrypted filesystems in such a way that you can use ssh-agent signing operations to unlock (mount) them – with no need to remember or enter a separate passphrase. Use it with ssh-add -c!

Note that the filesystem then remains mounted/decrypted until manually unmounted; ssh-agent is only consulted during the mount operation. The ssh-add -D command does not unmount the filesystem (analogously, it doesn’t close existing SSH sessions either). If you want screen-locking to umount these filesystems, consider the command:

umount -a -t fuse.encfs

That’s a slightly dangerous thing to do, though; it’s not guaranteed to work if any process is holding the filesystem open (by holding a file on it open, or having a current-directory inside it). Even if it does work, running processes might get very confused – perhaps, for instance, by writing important files into the unencrypted mount-point directory outside the EncFS, instead of inside the EncFS where you wanted them.

LATER EDIT: Extra bonus fifth SSH thing

TIL you can put

AddKeysToAgent confirm

in your .ssh/config and it’ll automatically do the equivalent of “ssh-add -c” for any key whose passphrase you supply to an ordinary ssh invocation. Semi-life-changing given the number of times I think my key is in ssh-agent, but it isn’t, and I end up having to enter the passphrase all over again...

Things I learned writing an SSDP implementation in Rust

Previously on #rust:

• Self-hosted CI for Rust and C++ using Laminar

A while ago I wrote Chorale, a collection of open-source media-library and networking software, mostly based on the UPnP (Universal Plug-’n’-Play) AV standards (but including other things as well, such as support for Rio Receiver and Empeg car-player devices, whose firmware I’d worked on at those companies, an even longer while ago).

Chorale never got much interest (or, as the tagline on this very blog puts it, “Waits for audience applause ... not a sossinge”); this is no doubt partly due to (closed) cloud streaming platforms for music hugely overtaking in popularity all dealings with (open) local media libraries, but no doubt also due to the completely zero amount of effort I actually spent promoting Chorale (or even, really, mentioning its existence).

So I’ve resolved to be a bit more talkative about open-source things I do from here on in. Inspired by a post by Simon Willison, I’m trying to adopt his mantra that anything you do, at least in the software world, isn’t finished until you’ve told people that you’ve done it – especially if part of the reason you’re doing it in the first place, is in the hope of spreading useful information and good practices.

In this post I’ll tell you what I did, when I went to implement one of the first, lowest-level parts of the UPnP protocol suite in Rust: SSDP. This is the backstory of the cotton-ssdp crate.

Background: a Simple Service Discovery Protocol

SSDP, the Simple Service Discovery Protocol, is aimed at making it easy for separately-purchased and separately-designed devices to discover each others’ resources over a local network, so that they can later interact using higher-level protocols such as UPnP itself. A resource might be a streaming-media server, or a router, or a network printer, or anything else that someone might want to search for or enumerate on a network. Like most protocols, there are two basic rôles involved: a server advertises the availability of a resource, and a client discovers available resources.

What precisely is advertised, or discovered, is, for each resource, a unique identifier for that particular resource (Unique Service Name, USN), an identifier for the type of resource (Notification Type, NT), and the location of the resource in the form of a URL.

SSDP is mainly used by UPnP systems, such as for media libraries and local streaming of music and video – it is how diskless media players such as the Roku Soundbridge discover the available music servers – but the mechanism is quite generic, and could as easily be used for any type of device or resource that must be discoverable over a network. Effectively it enables the automatic self-assembly of (locally-) distributed systems, including in ad hoc settings which don’t necessarily have expert network administrators close at hand to configure them manually.

For a learning project in Rust, SSDP is ideal as it’s about as simple as a network protocol could possibly be without being uninteresting. SSDP servers send out advertisements over UDP multicast to a “well-known” multicast group, which can be picked up passively by clients in that group; and clients can also actively send out search requests, also over UDP multicast, to which servers will respond.

There is no Internet RFC as such for SSDP – merely some expired drafts. The protocol is, instead, documented in the UPnP Device Architecture documents.

SSDP is very similar in motivation, and (it turns out) fairly similar in implementation, to the Multicast DNS (mDNS) protocol – but is not directly compatible.

Particular goals of the cotton-ssdp crate

In no specific order:

1. Produce a useful SSDP implementation, which other Rust users who need SSDP might reasonably hear about and decide to use;
2. Personally learn more about Rust and about the packaging, distributing, and publicising of Rust crates;
3. Implement both client and server;
4. Support use both in full-size/desktop/hosted environments, and in embedded/freestanding/no_std environments;
5. Do the Right Thing on multi-homed hosts (ones with multiple active network interfaces), which were the source of a lot of complexity in Chorale and in proprietary network protocols which I’ve worked on in my day job;
6. Follow, and indeed embody, if not evangelise, software engineering best practices: lots of documentation, lots of tests.

Things I learned about

1. Effective techniques for getting good test coverage
2. Automate your quality checks as far as possible
3. Not picking a lane when it comes to third-party ecosystems
4. Effective techniques for multi-homed hosts
5. The journey to no_std
6. Using the remarkable free tools available

1. Effective techniques for getting good test coverage

Coming from a C++ background, I was very impressed with the “batteries-included” nature of the Rust (Cargo) test setup. It’s as easy to start writing tests as it is to start writing the code in the first place – which is a Good Thing, as it means that every project can start out on the right side of history by including good testing from the very beginning.

But even here, there are better and worse ways to write tests, and it’s worth learning how to structure your tests in order to be most effective. As well as including the test harness itself, the Rust/Cargo system has baked into it the difference between unit tests and integration tests: outside the Rust world, this is a distinction about which philosophical debates can be had, but in Rust these two terms refer to two very specific and distinct things. When I started out in Rust, I was bundling all the tests into a mod tests in the same .rs file as the implementation: these are Rust’s unit tests. And I soon got in a pickle, because on the one hand I wanted to mock out (or dependency-inject) fake versions of some of the dependencies (specifically, system calls such as setsockopt whose failure is hard to arrange for in a test, but needs to be handled if it does happen in reality), but on the other hand I didn’t want the hassle of reimplementing the actual functionality of the real calls when testing the success case. The Rust mocking library mockall, for instance, replaces the real dependency completely when compiled in cfg(test) mode, as is done when compiling unit tests.

But Rust was way ahead of me here, and had already solved that problem. Tests for the success case – tests which can be written against the public API of the library – can be written as Rust integration tests, in the crate’s tests directory. Those are linked against the real version of the crate, compiled without cfg(test), so they get the actual, non-mock dependencies.

(As it stands, cotton-ssdp does not use mockall, but that’s because I had difficulty with “false negatives” in code-coverage, where it said not all regions were covered, but then didn’t show any actually non-covered regions when drilling-down into the source file. I suspect, but can’t prove, that this is because the coverage tool gets confused when a piece of code is monomorphised twice, once for real and once for mock, and can’t always correctly match up equivalent regions between the two.)

1.1 An example of dependency injection

Here’s the code I first wrote for creating a new UDP socket and setting the particular socket options that cotton-ssdp will need:

fn new_socket(port: u16) -> Result<mio::net::UdpSocket, std::io::Error> { let socket = socket2::Socket::new( socket2::Domain::IPV4, socket2::Type::DGRAM, None, )?; socket.set_nonblocking(true)?; socket.set_reuse_address(true)?; let addr = std::net::SocketAddrV4::new(std::net::Ipv4Addr::UNSPECIFIED, port); socket.bind(&socket2::SockAddr::from(addr))?; setsockopt(socket.as_raw_fd(), Ipv4PacketInfo, &true)?; let socket: std::net::UdpSocket = socket.into(); Ok(mio::net::UdpSocket::from_std(socket)) }

Now how does one test the error code paths from set_nonblocking and friends – all those question-marks? Well, one way is to use dependency injection – that is, to arrange that in unit-test builds, a “fake” version of the system-call is made (one which, for testing purposes, can be arranged to fail on demand), yet in the finished crate the actual, non-artificially-failing system-call is of course used. That change looks like this:

type NewSocketFn = fn() -> std::io::Result<socket2::Socket>; type SockoptFn = fn(&socket2::Socket, bool) -> std::io::Result<()>; type RawSockoptFn = fn(&socket2::Socket, bool) -> Result<(), nix::errno::Errno>; type BindFn = fn(&socket2::Socket, std::net::SocketAddrV4) -> std::io::Result<()>;   fn new_socket_inner( port: u16, new_socket: NewSocketFn, nonblocking: SockoptFn, reuse_address: SockoptFn, bind: BindFn, ipv4_packetinfo: RawSockoptFn, ) -> std::io::Result<mio::net::UdpSocket> { let socket = new_socket()?; nonblocking(&socket, true)?; reuse_address(&socket, true)?; bind( &socket, std::net::SocketAddrV4::new(std::net::Ipv4Addr::UNSPECIFIED, port), )?; ipv4_packetinfo(&socket, true)?; Ok(mio::net::UdpSocket::from_std(socket.into())) }   fn new_socket(port: u16) -> Result<mio::net::UdpSocket, std::io::Error> { new_socket_inner( port, || { socket2::Socket::new( socket2::Domain::IPV4, socket2::Type::DGRAM, None, ) }, socket2::Socket::set_nonblocking, socket2::Socket::set_reuse_address, |s, a| s.bind(&socket2::SockAddr::from(a)), |s, b| setsockopt(s.as_raw_fd(), Ipv4PacketInfo, &b), ) }

Now all the tests can call new_socket_inner, passing versions of new_socket, nonblocking, and so on which are as fallible as can be – but users of the real library are non the wiser; indeed, the API they see has not changed. (The actual tests are visible in commit d6cf9af2; they subsequently moved around a bit as later refactorings were done.)

This is also a standard dependency-injection technique in C++, but one of the hidden wonders of Rust is that the language design allows for a compiler to make lots of optimisations that, through technicalities, might be harder to prove correct in C++ (it’s not like I’ve quizzed the designers on whether this was an explicit goal of the design – but things have worked out that way so often, that really it must have been). The standard compiler is very good at optimising away abstractions like new_socket_inner: it spots that (in non-cfg(test) builds) the only call site of new_socket_inner is in new_socket and it inlines it completely; it also inlines all the lambdas (closures) and turns them back into direct function calls. In this case I checked, and the two versions of the code shown above produce identical assembler: a literally zero-cost abstraction.

Good code coverage is not, of course, any indicator of good code or even of good testing. But poor code coverage is very much an indicator of poor testing – so, much like a compiler warning, it’s always something worth looking at just in case it indicates a real underlying problem of poor code.

2. Automate your quality checks as far as possible

Engineering is the art of the repeatable; art is the engineering of the unrepeatable. Or, put differently: if it costs you actual human effort to make sure an engineered product, such as a particular version of a piece of software, is proper and correct, then that effort itself is, in a sense, written on the wind – as it tells no-one anything about whether any other versions of that software are proper or correct. So effort is usually much better applied on automating quality-assurance work than it is on performing (rote) quality-assurance work; it’s both dehumanising and non-cost-effective to send a person to do the job of a machine. (Though of course some quality-assurance work isn’t automatable: such work is often known as exploratory testing and works best when viewed as an exercise in creatively finding corner-cases or oversights in the automated test suite, which can then be automated in their turn.)

But there’s more to quality-assurance than testing. It’s also a discipline of quality-assurance to run code through analysis tools, to pick up on any potential errors. In the C++ world it’s long been received wisdom to enable most or all available compiler warnings, across multiple alternative C++ compilers if possible: the small iatrogenic downside of avoiding false positives from compiler warnings, is outweighed by the benefits accrued to the project each time a warning pinpoints an actual mistake. In Rust, compiler warnings are usually very good and actionable (and peculiar idioms for avoiding them seem rare), and Clippy lints even more so. The “cargo clippy” command could equally aptly have been called “cargo clear-up-my-misconceptions-about-how-this-language-works”, although “clippy” is definitely easier to type.

The principle of “the more automation the better” applies of course to continuous integration (automatic building and testing) too. An earlier post on this blog described how to set up a local CI system, though for packages that aren’t aimed at embedded systems, CI services are also available in the cloud (and some SaaS engineers speak as if the term “CI” implies a cloud-based service, having seen no other design). Projects hosted in public Github repositories can take advantage of Github’s CI infrastructure (and should do, even if only as a quality filter on incoming pull requests); more on that later on.

Once a crate’s functionality starts to be split up into Cargo “features”, configuring CI to test all feature combinations can be a chore. The cargo-all-features crate can help with this, by running “cargo build” (or check, or test) on all combinations of features in one command. This also helps to surface any issues where features are interdependent, but where that interdependency isn’t spelled out in Cargo.toml. (For instance, in cotton-ssdp, the “async” feature doesn’t compile without the “std” feature enabled, so Cargo.toml specifies that dependency.) Running “cargo test-all-features” takes, in theory, exponential time in the number of features – but, at least in cotton-ssdp’s case, the great bulk of the work is done just once, and the 2ⁿ-1 additional builds are all very fast.

At the top of cotton-ssdp’s lib.rs are the lines

#![warn(missing_docs)] #![warn(rustdoc::missing_crate_level_docs)]

which automate some quality checks on the documentation. Also assisting the documentation are these lines

#![cfg_attr(nightly, feature(doc_auto_cfg))] #![cfg_attr(nightly, feature(doc_cfg_hide))] #![cfg_attr(nightly, doc(cfg_hide(doc)))]

which, between them, arrange for those “only available with feature blah” labels on optional structs and methods in the docs.rs documentation – see later on for how to get docs.rs documentation builds to set the “nightly” attribute, as it isn’t built-in.

One last quality check, if you can call it that, is provided by Rust’s borrow-checker and ownership rules. A common refrain from Rust developers is, “Oh, I’m currently fighting the borrow-checker”. But what this reminds me of most, is an anecdote about a famous bull-fighter being asked what exercises he does to keep in shape for his toreadoring: “Exercises? There is something you don’t understand, my friend. I don’t wrestle the bull.” The anecdote appears in DeMarco and Lister’s “Peopleware” (and, slightly worryingly, apparently nowhere else on the whole of Google) – where the authors use it to suggest that if some activity feels like you’re wrestling a bull, you’re probably not going about it in the most efficient or masterful way. Hand-to-hand combat with the borrow-checker can sometimes be reframed as not heeding its suggestion that the design you’re pursuing could be done differently with more clarity around memory-safety, and thus better reliability or maintainability. The borrow-checker is like a compiler warning, but for your design instead of your implementation, and it’s actionable in the same way: it’s telling you that you might be able to barrel on regardless, but you should really stop and think about whether you’re doing things the best way.

Though admittedly, by that theory, it’s a shame that the borrow-checker only offers that suggestion at quite a late stage of the design process (viz., during actual coding); it’s a bit like being told, “You can’t lay that next brick there.” – “Why not?” – “Because the house that you’re building is going to be ugly.” It would have been better if the ugliness of the house had been spotted at the blueprint stage! There’s a gap in the market, and indeed in the discourse as a whole, for a book on Rust design patterns which would help stave off these issues at an earlier conceptual stage.

3. Not picking a lane when it comes to third-party ecosystems

There are two, not very compatible, ways of writing Rust networking code: synchronous and asynchronous. When hoping to write a library that’s useful to people (one of the stated goals), it makes most sense to me to try and be useful on both sides of this divide. After all, SSDP itself can be boiled down to, packets arrive, other packets go out in response (or in response to a timer expiring). That’s a usefully narrow-sounding interface. So what cotton-ssdp does, is hide all the actual SSDP work away in a struct called Engine, and offer two as-thin-as-possible wrappers over it: one for the synchronous (mio) case (Service) and a second for the asynchronous (tokio) case (AsyncService).

This also involved abstracting over the difference between tokio::net::UdpSocket and mio::net::UdpSocket, using traits which are implemented for both and over which the relevant methods of Engine are generic:

/// Notify the `Engine` that data is ready on one of its sockets pub fn on_data<SCK: udp::TargetedSend + udp::Multicast>( &mut self, buf: &[u8], socket: &SCK, wasto: IpAddr, wasfrom: SocketAddr, )

Both the Engine and UDP abstractions will hopefully also turn out to be exactly what’s needed when porting SSDP to embedded platforms using smoltcp or a similar embedded networking stack. (So in fact, it’s not quite right to say that the SSDP implementation is “hidden” away in Engine – Engine is part of the public library API.)

4. Effective techniques for multi-homed hosts

Networking applications that run on multi-homed hosts – machines with more than one TCP/IP-capable network interface – often need to take particular care over which networks services are offered to: it’s commonplace for a host such as a firewall to have interfaces onto networks with different security policies. (This is so rarely dealt with well, that it’s also commonplace for such setups to resort to containerisation to ensure that certain services are only offered on certain interfaces – which does have the advantage that all such containers or services can then have their network access configured in the same way, rather than with service-specific ways.)

But supporting multi-homed hosts well in a networking server isn’t a huge challenge, so long as it’s thought-about early in the design process. The fundamental idea is always to keep track of which network interface is being used to talk to any given client; Linux, for instance, assigns long-lived “interface indexes” (smallish integers) to each available hardware interface. Knowing which interface packets arrived on (especially in the UDP case) gives the server enough information to know how to send its reply – and in the particular case of a discovery protocol such as SSDP, can inform the content of the reply.

For example, if the SSDP resource being advertised is, from the server’s point-of-view, http://127.0.0.1:3333/test, and its Ethernet IP address is 192.168.1.3, and its wifi IP address is 10.0.4.7, then anyone listening to SSDP on Ethernet should be told that the resource is at http://192.168.1.3:3333/test and anyone listening on wifi should instead be told http://10.0.4.7:3333/test.

In a UDPv4 service such as SSDP, the Linux IP_PKTINFO facility can be used to obtain, at recvmsg time, the interface index on which a packet arrived; similarly, sendmsg can be told which interface to send on. (Fortunately, unlike SO_BINDTODEVICE, this does not require CAP_SYS_RAW, although that restriction on SO_BINDTODEVICE was lifted in kernel 5.7.) The SSDP server can then determine which of the host’s IP addresses that particular client should use, and rewrite the resource URL accordingly when replying.

With care, this can even be used to do the Right Thing when more than one network interface has the same IP network and netmask, just so long as they still have different IP addresses – a situation which is particularly likely to arise if RFC3927 “Zeroconf” networking is in use on multiple physical networks.

Although cotton-ssdp is currently IPv4-only, it appears that IPv6 has noticed and ameliorated the same problem, with APIs taking IPv6 link-local addresses also taking a “Scope ID” which is intended to be used very similarly to this use of the interface-index.

(Every time I see “multi-homed” written down, even if I wrote it myself, I read it the first time as “multi-horned”, which is a metaphor so much more apposite that it makes me wonder whether “homed” was just a typo that stuck.)

5. The journey to no_std

Rust draws a formal distinction between (what in the C world are called) hosted and freestanding implementations: hosted implementations are servers, desktops, and mobile platforms with the usual wide range of standard library facilities, and freestanding implementations are embedded systems where even standard library facilities are not necessarily available. (In fact C draws this distinction too, but as the standard C library is itself so low-level, only the most deeply-embedded systems are freestanding by C’s definition.)

Rust target platforms indicate their freestanding nature with “-none-” as the OS part of their target triple, such as in “thumbv7em-none-eabi”; Rust crates targeting such platforms indicate that by a crate-level #![no_std] attribute. In a no_std crate, much of the standard library is still available (very core parts such as Result and Option are in a crate called core, which works even if no heap is available; facilities that require an allocator live in a second crate called alloc), but anything which requires system-calls is not available.

As cotton-ssdp is aimed at both hosted and embedded users, it needs to be possible (but optional) to compile it in no_std mode. The standard idiom for this, is to have a (Cargo) feature called std, which is usually enabled by default, but can be disabled to produce a no_std-compatible crate. This is taken care of in the code by adding, at the top of lib.rs, the line...

#![cfg_attr(not(feature = "std"), no_std)]

...and then in Cargo.toml, declaring std as a feature and making any non-no_std-compatible dependencies optional and dependent on std; the Cargo.toml changes are in commit 47a0290d, which also marks the examples and integration-tests as depending on std.

The issues that might arise when enabling no_std mode in a crate revolve around: system types whose names (libraries) are different; system types which aren’t available at all; dependencies which themselves need to have their no_std build enabled; and dependencies which have no no_std mode, which must be either encapsualated away or worked around altogether. All of those issues cropped up in cotton-ssdp.

The entire journey to no_std support can be seen in the pull request, though it might look a bit odd because in fact I only created the PR retrospectively, after the branch was already merged, for the purposes of this blog post; that seemed the easiest way to get all the commits listed in one place. The eleven commits on that branch are (in chronological order):

1. 364e7c5 Split UDP implementations into core, std, mio, tokio – once again, something seemed a bit tiresome (annotating everything in udp.rs with cfg(feature = "std")) but that tiresomeness actually represented a “compiler warning” about the design; the UDP traits belong in a separate source file to the std-using implementations, and they in turn belong in separate source files to each other. Now only the mod declarations need to have cfg annotations attached!
2. e590562 Make message.rs no_std-compatible – some of the std facilities previously used here are available in core or alloc, but HashMap is not (because it relies on a random number generator), and nor is io::Cursor (as it uses io::Error, which is in std because it needs to be OS-specific in order to represent system-call errors). Fortunately, reimplementing Cursor from scratch is not hard.
3. 0380edf Make UDP errors more ergonomic (and also more no_std-compatible) – rather than forcing SSDP errors into the io::Error template, use a custom error class (preparatory to cfg-ing out io::Error altogether). This actually makes the crate’s errors easier to use, by following the suggestions in in a recent blog post I read about modularising errors.
4. 47a0290 Featureise std, sync, and async – this is the above-mentioned commit which actually introduces the cargo-level features. As well as enabling no_std, making “async” a feature means that clients who need only the synchronous version, do not need to pull in the sizeable tokio dependencies. This commit also adds the no_std_net crate, which usefully provides the common IP address types in no_std builds, while being a pass-through for the standard-library ones in std builds. (A future release of Rust will move those types to core, but I didn’t want to wait.)
5. fc344f0 Update dependencies – fixes test breakage from the previous commit, as cotton-ssdp’s tests require cotton-netif’s sync feature, not just its std one. Fortunately, using cargo test-all-features made this issue show up straightaway.
6. 0005fe0 Factor RefreshTimer out of Engine – it’s desirable for Engine to compile in no_std mode, because that’s intended to be the basis even of embedded implementations of SSDP. But Engine used std::time::Instant, which (especially in the now method) relies on system-specific clock facilities (std::time::Duration does not rely on those facilities, and is available in no_std builds as core::time::Duration). In practice, most embedded platforms support some variant of rtic_monotonic::Monotonic for clock facilities (a list of implementations of that trait for common microcontrollers can be found on crates.io), but abstracting over both that and the standard library felt like an abstraction too far, so instead this commit moves the whole refresh-timer mechanism out into a separate struct, so that an embedded implementation depending on Monotonic can be provided later and just slotted-in.
7. 6899933 Actually set no_std for non-feature-"std" – this is where the crate is first built with no_std set, which necessitated use of the alloc crate to find String and Format, as well as cfg’ing off some uses of std that couldn’t be ported. In particular, the variant of the udp::Error enum which wraps an underlying system-call error is not, of course, no_std. The [#non_exhaustive] tag tells the Rust compiler that more variants (perhaps one encapsulating a smoltcp error) may be added in future.
8. 5ac330e Remove "url" crate from public API – this crate, a hard dependency used within Engine (for rewriting resource URLs on a per-IP-address basis, see above) which does not compile under no_std. Fortunately only a tiny fraction of the crate’s functionality is actually used in cotton-ssdp – functionality which can, as the least-worst option, just be reimplemented from scratch. But the first thing to do is eliminate the url::Url type from the public interface; this is a breaking change which would cause a semver major change, if it weren’t for the fact that the semver major already needed changing due to other API changes since 0.0.1.
9. 38694d1 Remove usage of "url" crate – this is where the url.set_ip_host call is replaced with a simpler equivalent. Notably this new code has a bug, only fixed later in b92ded1, which causes it to do the Wrong Thing with any URL containing an explicit port (such as, er, the example ones discussed above where the need for the rewriting is explained).
10. 261a6bb Replace HashMap usage with BTreeMap – as mentioned, HashMap is not available in no_std, but BTreeMap is, and has essentially the same interface.
11. 0faacd5 Actually compile ssdp including Engine with no_std – finally cargo build --no-default-features works (including Engine and Message). Moreover, so does cargo build --no-default-features --target thumbv7em-none-eabi – that’s a freestanding target with no std support whatsoever.

Oh, plus there’s another commit that wasn’t on that branch, but should have been:

12. b92ded1 Fix rewrite_host bogusly stripping ":port" from URLs – fixes a bug introduced when removing the “url” crate. It seems pretty likely that the only forms of URL that make any sense as SSDP locations start with http://127.0.0.1..., not even HTTPS – but the Principle of Least Surprise suggests that the code should at least not flake out if given one in a different form.

Between them, those commits hopefully tell the whole story of taking a crate initially written in hosted Rust, and making it work in both hosted and freestanding applications.

6. Using the remarkable free tools available

Most of the anecdotes in this post are hopefully applicable to both proprietary and open-source software development. But in the open-source case in particular, it’s worth looking into the freely-available online tools that support such development efforts. Even getting started in Rust development introduces everyone to crates.io and to docs.rs, but it’s also worth looking into:

Github Actions cloud CI

Continuous-integration servers run by Github, and automatically triggered on pushes to Github repositories (if your project contains YAML files for workflows, perhaps a bit like cotton’s one).

docs.rs

Documentation pages on docs.rs are built automatically for all crates published to crates.io. The build is always done with a nightly toolchain, so you can use nightly-only features such as doc_auto_cfg – just make sure you leave your crate buildable by stable and other non-nightly toolchains, perhaps by guarding the doc_auto_cfg attribute like this:

#![cfg_attr(nightly, feature(doc_auto_cfg))]

and then adding this to your Cargo.toml such that docs.rs builds always enable the “nightly” cfg:

[package.metadata.docs.rs] all-features = true rustdoc-args = ["--cfg", "nightly"]

deps.rs

Checks the dependencies in your Cargo.toml, and warns you about out-of-date versions (and warns you strongly about ones with known security problems). It’s wired up to Github, so can be used on any project published there. (For Cotton, that means there’s one page for the whole monorepo, not one page per crate, but it’s subdivided into the dependencies of each crate.)

codecov.io

Code-coverage as a service. You have to sign up for an account, but it’s free for public repositories. Support can be incorporated into Github CI workflows, which is how it works for cotton-ssdp.

The neat thing is that many of these services provide “badges”: little status icons that you can add to README.md on Github, or to any web page. The statuses below aren’t just a pretty picture: each one is the live status, on most of them you can click-through to the underlying data, and as you’re reading them right now, you might be noticing an issue before I do:

Cotton project:
cotton-netif:
cotton-ssdp: