Undefined Behavior

Fiber Cleaner Test

Hotaru, Error, jo — Sun, 08 Dec 2024 12:20:00 +0100

Intro

This post is a collaboration between Hotaru, error and jo. It is published at swagspace.org as well as undefinedbehavior.de.

The idea behind this post initially came from a talk at DENOG16 ¹ about problems with return loss in optical transmission. Our question was: How dirty do fibers get and how much work is it to get them clean again.

Making fibers dirty

We primarily tested two different kinds of contamination. The first kind is touching the fiber. The second kind is contamination with dirt, e.g. from the floor. Both kinds can happen easily when handling fiber.

It is interesting to see how visually distinct the different kinds of contamination are. Just touching the polished face of the fiber once with our fingers covered the whole face with a thin grease film.

Swiping over the face of the fiber leads to a more streaky pattern.

The contamination when touching hair is very similar to touching the fiber with your fingers.

Dropping the fiber on the floor (in our case a carpet) contaminates the fiber with many small pieces of dirt.

Different types of cleaners

Click-Cleaners

Click-Cleaners are somewhat pen shaped and fit directly on top of the connector of the fiber. By pushing down on them, a small cleaning thread is dragged over the face of the fiber, cleaning it in the process. Afterwards, a spring pushes the cleaner back into it's original position, ready to be used again.

Manual swiping

Manual swiping cleaners have a button that exposes a piece of cleaning cloth when pressed. The fiber can then be swiped over the cloth to clean it. When the button is released, the cover closes and the microfiber cloth moves forward.

Miscellaneuos "cleaning utensils"

Sometimes you have to improvise. What if you don't have professional cleaning utensils at hand? So we looked at wether a pair of work trousers and a piece of cleaning cloth for glasses work, as well as unicorn Ida and the locomotive Chuff Chuff.

Tests

We did several tests with each type of cleaner and different kinds of contamination. We cleaned until the fibers passed IEC 61300-3-35 testing. The standard defines 4 areas of the fiber. The core (A), cladding (B), a small glue area (C) and the contact area of the ferrule (D). The test specification allows for some minor defects on the fiber, especially in the contract area. So there might be some contamination left. Keep this in mind when looking at the figures.

We used an EXFO FIB430B video microscope with automatic analysis features to analyse the fibers.

All our tests were done on Singlemode LC/UPC connectors.

Cletop S

The Cletop S is based on manual swiping. In all 4 tests the Cletop S cleaned the fiber after one swipe.

Senko Smart Cleaner

The Senko Smart Cleaner is of the click cleaner variety. In our tests we had the issue, that this cleaner did not reliably clean the fiber. It looks like some part of the fiber did not get wiped at all.

This is an example of a dirty fiber before cleaning

And this is the result of the first cleaning.

We have seen this pattern multiple times with this device during our tests. 3 clicks were necessary in order to reliably clean the fibers.

IBC Brand Cleaner Zi125

This cleaner is also of the click cleaner variety. In all but 1 of our 4 tests this cleaner cleaned the fiber in one click. In the one attempt that didn't clean the fiber, we observed a piece of contamination being moved onto the fiber that wasn't there before.

This is the uncleaned fiber. [°Note: This fiber passed the IEC test, because the contamination is small enough.] After cleaning this fiber a large piece of contamination was moved into the contact area that was not on our fiber before (or at least not in the part that we were able to observe with out microscope).

Here we can see a large piece of contamination in the top center. Cleaning it once again removed the defect.

This shows us, that it is possible to introduce new defects through cleaning.

Other cleaning utensils

We had several other "improvised cleaning utensils" at hand so we did a couple of quick tests. Hold on tight, we are entering shitpost territory now.

Work trousers and microfiber cloths for glasses

We did a quick test on a pair of work trousers that were worn for several days. The results were not great.

The trousers were not able to get the fiber clean, not even after several swipes. The other issue with potentially contaminated pieces of clothing is, that you might introduce new contamination.

We tried the same with used microfiber cloth for glasses.

Technically this fiber is clean according to the standard. However there is a significant chance, that the defects still on the fiber end up in the center of the fiber. So we can't recommend this method.

Plushies

In case you are carrying your plushy companion into the datacenter or the field, they might help you cleaning fibers. We did tests with multiple different kinds of surfaces on Chuff-Chuff, a locomotive plushie. Unfortunately none produced good results.

However the results that were produced by unicorn Ida were surprisingly good.

Ida got the fiber almost clean with the first swipe:

There is only one piece of contamination. Sadly that one is directly in the center of the fiber where no contamination is acceptable. Further swipes were not able to remove this contamination. (A purpose-built cleaner did it first try).

Conclusions

Making fibers dirty is surprisingly easy. Simply dropping them on the ground or touching them is sufficient.
The Cletop S and IBC Brand Cleaner usually cleaned the fiber in 1 swipe/click, 2 at most.
If you are not working on very critical optical interfaces 1 swipe on a cletop or 2 clicks from a good cleaner are sufficient.
It doesn't occur frequently, but it is possible to reintroduce dirt to the contact area of a fiber from other parts through cleaning. If you are working on critical patches, use a scope to verify your work.
Keep in mind, that this is a limited test on LC-PC/UPC connectors only and that we are only testing the patch cable end of the interface. The results might not hold for other types of connectors like E2k, SC or the APC variants of these connectors.
Due to missing equipment we were not able to test the cleaning performance of the click cleaners in the recepacle part of the connection, e.g. a transceiver.

T. Nakagawa. (2024, November) Return loss problems associated with faster optical networks. https://media.ccc.de/v/denog16-53304-return-loss-problems-associated-with-faster-optical-networks ↩

git commit --vandalism

Error — Thu, 25 May 2023 18:55:00 +0200

This is not a vulnerability report. I intentionally disabled safeguards in the code in order to be able observe this behavior. However, it's a nice example for why you shouldn't use SHA-1 for anything security related and a good opportunity to learn more about the inner workings of git.

In 2017, the Cryptology Group at Centrum Wiskunde & Informatica (CWI) and the Google Research Security, Privacy and Anti-abuse Group announced the first (public) SHA-1 hash collision. They generated 2 PDF files with the same hash in an attack they called SHAttered. While it still required a lot of processing power (the equivalent of 6,500 years single-CPU computations and 110 years of single-GPU computations), it showed that such attacks are not only theoretically possible - they are technically feasible. ¹

IDs in git, for example for commits, are generated using SHA-1 by default. As explained by git's creator, Linus Torvalds, the hash function is not used for security. It's simply used to generate a checksum, like e.g. a CRC.[°The quote "We check a checksum that's cryptographically secure. Nobody has been able to break SHA-1" didn't hold up that well, but in general, Linus' point still stands: git uses SHA-1 for consistency checks, not for security.] ² But what happens if a collision occurs, or someone intentionally causes a collision? Let's try it out.

So if the SHAttered files already have a hash collision, they could be used to cause a collision in git, right? Let's put them in a repository and find out.

$ wget https://shattered.io/static/shattered-1.pdf -O shattered1.pdf -q
$ sha1sum shattered-1.pdf
38762cf7f55934b34d179ae6a4c80cadccbb7f0a  shattered-1.pdf
$ wget https://shattered.io/static/shattered-2.pdf -O shattered2.pdf -q
$ sha1sum shattered-2.pdf
38762cf7f55934b34d179ae6a4c80cadccbb7f0a  shattered-2.pdf
$ diff shattered-1.pdf shattered-2.pdf
Binary files shattered-1.pdf and shattered-2.pdf differ

As promised, these files have different contents, but the same SHA-1 hash. The next step is to add them to a git repository and observe what happens.

$ cp shattered-1.pdf shattered.pdf
$ git add shattered.pdf
$ git commit -m 'shattered'
[main (Root-Commit) 0aa8c5a] shattered
1 file changed, 0 insertions(+), 0 deletions(-)
create mode 100644 shattered.pdf
$ cp shattered-2.pdf shattered.pdf -f
$ git add shattered.pdf
$ git commit -m 'shattered'
[main f1906a7] shattered
1 file changed, 0 insertions(+), 0 deletions(-)

Looking at the result, this did not seem to cause any issues. If the commit ID is a SHA-1 hash, shouldn't this cause a commit ID collision? Such a collision did not happen. The file was updated successfully in the second commit and checking out the first commit restores the original version. Investigating this closer shows that git somehow calculates different hashes for these files.

$ git hash-object shattered-1.pdf
b621eeccd5c7edac9b7dcba35a8d5afd075e24f2
$ git hash-object shattered-2.pdf
ba9aaa145ccd24ef760cf31c74d8f7ca1a2e47b0

Internally, git is a key-value store. It generates an ID for each object that it stores, which can later be used to identify and retrieve the object. By default, this ID is a SHA-1 hash. The input given to the hash function depends on the object type. For files, which are stored as so called blobs, the ID is generated by hashing the string blob, the length of the file, a null byte and the file itself. ³

$ echo "so long and thanks for all the fish" > test.txt
$ git hash-object test.txt
8b86cb67f1f19db567a100b55edb5466a33e7fb7
$ printf "blob $(wc -c <test.txt)\0$(<test.txt)\n" | sha1sum
8b86cb67f1f19db567a100b55edb5466a33e7fb7  -

Blobs can be assigned names to using tree objects. These objects can group multiple blobs and trees, which are referenced by their id. ³ A tree object could look like this for example:

$ git cat-file -p 8004c8a7b6fce1452539556bb4c4c91b92b5c2bc
100644 blob ba9aaa145ccd24ef760cf31c74d8f7ca1a2e47b0    shattered.pdf

However, one does not usually work directly with blobs and trees. It's what git uses internally, but from a user perspective, it's usually not necessary to interact with git on that level. The usual workflow consists out of editing files, staging these changes and then committing them. Commits are just another object type in git. These objects associate a tree with an author and committer,[°The author and the committer can be different. The author is who wrote the code and the committer is who added it to the repository.] a parent commit (if it exists) and a commit message to describe to changes included in this commit. ³ The resulting object can once again be viewed with git cat-file.

$ git cat-file -p f1906a7bf1c4c8ce49f264e2f3ba313c305b7ede
tree 8004c8a7b6fce1452539556bb4c4c91b92b5c2bc
parent 0aa8c5ad0be568d680d5f613807bde46a39424e3
author error <redacted> 1683729684 +0200
committer error <redacted> 1683729684 +0200

shattered

The commit ID is generated by hashing this information, prepended by the same header that is used for the other object types: $object-type $length\0.[°You can find a nice example how the commit hash is constructed here.]

OK, so that's the reason why git hash-object returns different values for the two SHAttered files.[°Try it yourself if you want: prepending any data to these files will make their SHA-1 hashes differ. echo test | cat - shatterd-1.pdf | sha1sum results in a different hash than echo test | cat - shatterd-2.pdf | sha1sum] Further, this explains why the commit hashes didn't collide. There won't be any progress made by using the existing SHA-1 hash collision from SHAttered, but collisions can still happen. It's unlikely to find one by accident and it's expensive to cause them intentionally, but it's not impossible. How would git behave in this case? Let's vandalize the code a little bit and find out.

Git can be built with multiple different SHA-1 backends. By default, it uses an implementation with a collision attack detection mechanism.[°This collision attack detection is not relevant for our experiment for now. We'll get back to it later.] It can be found in the SHA1DC directory.

int SHA1DCFinal(unsigned char output[20], SHA1_CTX *ctx)
{
    uint32_t last = ctx->total & 63;
    uint32_t padn = (last < 56) ? (56 - last) : (120 - last);
    uint64_t total;
    SHA1DCUpdate(ctx, (const char*)(sha1_padding), padn);

    total = ctx->total - padn;
    total <<= 3;
    ctx->buffer[56] = (unsigned char)(total >> 56);
    ctx->buffer[57] = (unsigned char)(total >> 48);
    ctx->buffer[58] = (unsigned char)(total >> 40);
    ctx->buffer[59] = (unsigned char)(total >> 32);
    ctx->buffer[60] = (unsigned char)(total >> 24);
    ctx->buffer[61] = (unsigned char)(total >> 16);
    ctx->buffer[62] = (unsigned char)(total >> 8);
    ctx->buffer[63] = (unsigned char)(total);
    sha1_process(ctx, (uint32_t*)(ctx->buffer));
    output[0] = (unsigned char)(ctx->ihv[0] >> 24);
    output[1] = 1;  //(unsigned char)(ctx->ihv[0] >> 16);
    output[2] = 1;  //(unsigned char)(ctx->ihv[0] >> 8);
    output[3] = 1;  //(unsigned char)(ctx->ihv[0]);
    output[4] = 1;  //(unsigned char)(ctx->ihv[1] >> 24);
    output[5] = 1;  //(unsigned char)(ctx->ihv[1] >> 16);
    output[6] = 1;  //(unsigned char)(ctx->ihv[1] >> 8);
    output[7] = 1;  //(unsigned char)(ctx->ihv[1]);
    output[8] = 1;  //(unsigned char)(ctx->ihv[2] >> 24);
    output[9] = 1;  //(unsigned char)(ctx->ihv[2] >> 16);
    output[10] = 1; //(unsigned char)(ctx->ihv[2] >> 8);
    output[11] = 1; //(unsigned char)(ctx->ihv[2]);
    output[12] = 1; //(unsigned char)(ctx->ihv[3] >> 24);
    output[13] = 1; //(unsigned char)(ctx->ihv[3] >> 16);
    output[14] = 1; //(unsigned char)(ctx->ihv[3] >> 8);
    output[15] = 1; //(unsigned char)(ctx->ihv[3]);
    output[16] = 1; //(unsigned char)(ctx->ihv[4] >> 24);
    output[17] = 1; //(unsigned char)(ctx->ihv[4] >> 16);
    output[18] = 1; //(unsigned char)(ctx->ihv[4] >> 8);
    output[19] = 1; //(unsigned char)(ctx->ihv[4]);
    return ctx->found_collision;
}

In order to cause collisions between object IDs in git, it is helpful to reduce the size of the identifier. To achieve this, every byte of the hash after the first one was set to 1. This reduces the effective length of the identifier to 1 byte. 256 values may still sound like a lot, but there's no need to check every possible value. We're not looking for a specific hash, any collision will do. The first collisions should occur after a few tries.[°Such collisions are already very likely after a surprisingly small amount of attempts if the value range is not too large. Look up the birthday problem if you don't know it.]

So let's compile git with our "improvement" and try to cause some collisions.

$ touch test
$ ../git/git add test
$ ../git/git commit -m 'test'
[main (root-commit) c101010] test
1 file changed, 1 insertion(+)
create mode 100644 test
$ touch test2
$ ../git/git add test2
$ ../git/git commit -m 'test2'
[main db01010] test2
1 file changed, 0 insertions(+), 0 deletions(-)
create mode 100644 test2
$ ../git/git log
commit db01010101010101010101010101010101010101 (HEAD -> main)
Author: error <redacted>
Date:   Sun May 7 20:32:54 2023 +0200

    test2

commit c101010101010101010101010101010101010101
Author: error <redacted>
Date:   Sun May 7 20:32:00 2023 +0200

    test

The fist two commits may not collide, but it's already obvious what our little modification to the source code did: only the first 2 characters of the commit hash differ, everything afterwards is filled with the same two characters.

Continuing to add and commits to this repository, the first collision was observed on the 7th attempt.

$ touch test7
$ ../git/git add test7
$ ../git/git commit -m 'test7'
fatal: 0f01010101010101010101010101010101010101 is not a valid 'tree' object
$ ../git/git cat-file -p 0f01010101010101010101010101010101010101
tree 8501010101010101010101010101010101010101
parent e801010101010101010101010101010101010101
author error <redacted> 1683484473 +0200
committer error <redacted> 1683484473 +0200

test5

The collision happened between the tree object for the new commit and an already existing commit. Interestingly, git did notice this problem only when it tried to add the tree to the commit and found an object of a wrong type instead. The original object was not modified in the process. It looks like the attempt to add a new object failed silently.

On the 15th attempt, another interesting collision occurred:

$ touch test15
$ ../git/git add test15
$ ../git/git commit -m 'test15'
[main e201010] test12
1 file changed, 0 insertions(+), 0 deletions(-)
create mode 100644 test12
$ ../git/git log
commit e201010101010101010101010101010101010101 (HEAD -> main)
Author: error <redacted>
Date:   Sun May 7 21:12:38 2023 +0200

test12

commit 3b01010101010101010101010101010101010101
Author: error <redacted>
Date:   Sun May 7 21:12:06 2023 +0200

test11

commit 6001010101010101010101010101010101010101
Author: error <redacted>
Date:   Sun May 7 21:11:40 2023 +0200

test10

In this case both of the colliding objects are commits. This time, git did not even print an error message. After the creation of the new commit object once again failed silently, git proceeded to check out the existing commit with the same hash.[°People assume that a commit history is a strict progression of cause to effect. But actually from a non-linear, non-subjective viewpoint it's more like a big ball of wibbly-wobbly timey-wimey... stuff.] This confirms the assumption that git will keep the existing object in case of a collision. The other commits that got rolled back are still present as objects in git and can be checked out, but especially if the user does not notice what happened here and continues working, this can seriously mess up the commit history.

Those tests were continued until the following collisions occurred and git's behavior could be observed:

Collisions between two blobs: Creating the new blob object fails silently. The existing blob object remains unchanged.
Collisions between blobs and trees: Creating the new blob object fails silently. The existing tree object remains unchanged.
Collisions between blobs and commits: Creating the new blob object fails silently. The existing commit object remains unchanged.
Collisions between blobs and tags: Creating the new blob object fails silently. The existing tag object remains unchanged.
Collisions between trees and blobs: Creating the new tree object fails silently. The existing blob object remains unchanged. Trying to commit this results in an error message complaining that the object is not a valid tree object.
Collisions between two trees: Creating the tree object fails silently. The existing tree object remains unchanged. Trying to commit this, git commits the old tree again. This effectively means a rollback to the commit associated with of the old tree while keeping the commit history.
Collisions between trees and commits: Creating the new tree object fails silently. The existing commit object remains unchanged. Trying to commit this results in an error message complaining that the object is not a valid tree object.
Collisions between trees and tags: Creating the new tree object fails silently. The existing tag object remains unchanged. Trying to commit this results in an error message complaining that the object is not a valid tree object.
Collisions between commits and blobs: Creating the new commit object fails. The existing blob object remains unchanged. Git attempts to check out the new commit (that wasn't created) and fails with the error message fatal: cannot update ref 'refs/heads/main': trying to write non-commit object $HASH to branch 'refs/heads/main'.
Collisions between commits and trees: Creating the new commit object fails. The existing tree object remains unchanged. Git attempts to check out the new commit (that wasn't created) and fails with the error message fatal: cannot update ref 'refs/heads/main': trying to write non-commit object $HASH to branch 'refs/heads/main'.
Collisions between two commits: Creating the new commit object fails silently. The existing commit remains unchanged and is checked out.
Collisions between commits and tags: Creating the new commit object fails. The existing tag object remains unchanged. Git attempts to check out the new commit (that wasn't created) and fails with the error message fatal: cannot update ref 'refs/heads/main': trying to write non-commit object $HASH to branch 'refs/heads/main'.
Collisions between tags and blobs: A new file is created under .git/refs/tags pointing at the hash, but the creation of the new tag object under .git/objects fails silently. The existing blob object remains unchanged.
Collisions between tags and trees: A new file is created under .git/refs/tags pointing at the hash, but the creation of the new tag object under .git/objects fails silently. The existing tree object remains unchanged. The tag is displayed by git tag -l, but attempts at checking it out fail with the error message fatal: Cannot switch branch to a non-commit.
Collisions between tags and commits: A new file is created under .git/refs/tags pointing at the hash, but the creation of this new tag object under .git/objects fails silently. The existing commit object remains unchanged. The tag is displayed by git tag -l, but is interpreted as a lightweight tag for the colliding commit.
Collisions between two tags: A new file is created under .git/refs/tags pointing at the hash, but the creation of this new tag object under .git/objects fails silently. The existing tag object remains unchanged. Since the new tag reference object points at the old tag object, the new tag will be an alias for the old tag, with the same message and object reference.

Git does not overwrite existing objects. Creating a new object with a hash that's already associated with another object always fails silently. However, depending on the object type, git shows some interesting behavior. If the object type fits, git will just continue with it as if nothing is wrong, e.g. it will commit an old tree or checkout an old commit. Git only responds with an error if the object types are incompatible, e.g. if the collision causes a reference to a tree to point at a tag object instead.

Hash collisions, intentional oŕ not, can become a problem for git. To mitigate this risk, git includes a mechanism that detects SHA-1 collision attacks and reacts by hashing the suspected block 3 times, extending SHA-1 from 80 to 240 steps in these cases. This ensures that different hashes are generated in theses cases.⁴

Further, git does not only support SHA-1. It supports SHA-256, too. Unlike SHA-1, SHA-256 is considered cryptographically secure.

Actual SHA-1 collisions are very unlikely to occur as a coincidence. The collisions in this experiment could only be observed after the code was modified to limit the effective hash size to 1 byte. Otherwise, it would not have been possible to create collisions with the available resources. However, SHAttered shows that it is feasible to intentionally cause such a collision.[°Whether or not attackers with the resources required to do this are part of your threat model is up to you to decide.]

In case of a collision, git shows some interesting behavior that may not be noticed immediately. This may leave the git repository in an unintended state. Further, attackers could modify files and commits by replacing objects with specifically crafted objects that produce the same hash. This risk is mitigated by the use of the collision attack detection mechanism.

Even though the risk is already mitigated, this example shows why SHA-1 should not be used for cryptographic hashing and was a good opportunity to learn more about git itself. If you want to try it out yourself, you can have a look at the code with my modification here.

M. Stevens, E. Bursztein, P. Karpman, A. Albertini and Y. Markov (2017, February) "The first collision for full SHA-1", https://shattered.io/static/shattered.pdf. ↩
L. Torvalds (2007, May) "Tech Talk: Linus Torvalds on git", https://www.youtube.com/watch?v=4XpnKHJAok8&t=56m20s. ↩
S. Chacon, B. Straub et al. (2014) "10.2 Git Internals - Git Objects" in Pro Git, https://git-scm.com/book/en/v2/Git-Internals-Git-Objects. ↩↩↩
M. Stevens, D. Shumow (2017) "sha1dc/sha1.h", https://github.com/git/git/blob/master/sha1dc/sha1.h. ↩

RFCartography - Visualizing relations between RFCs

Error — Tue, 28 Feb 2023 19:50:00 +0100

RFCs are nice documents which explain how the internet works. They define protocols, describe best practices and discuss the organisation of internet infrastructure. Unlike some standards from other organizations, they are publicly available for free, usually written in a comprehensible manner and structured in a logical way.

Looking for a new project to do in my spare time at the end of 2022, I ended up thinking about how to visualize relations between RFCs. I always liked the way RFCs are connected to each other: Information about which RFC updates which, which RFC is obsoleted by which, and so on, all displayed at the top of the document, easy to find. So I decided to create RFCartography[°portmanteau word built out of "RFC" and "Cartography"; If you were under the impression that it has something to do with radio frequencies, you were misled, sorry.], a small tool that draws graphs of these relations.

Parsing all the RFCs in order to extract the required information would have been a lot of effort and probably prone to errors. Fortunately, rfc-editor.org has a nice, machine readable RFC index file in XML format with all the meta data for each RFC. ¹ This includes, among other data, all the relations between RFCs:

obsoletes: points to older RFCs which are superseded by this document ², ³
obsoleted-by: points to newer RFCs which supersede this document ², ³
updates: points to older RFCs which are modified and/or extended by this document ², ³
updated-by: points to newer RFCs which modify and/or extend this document ², ³
is-also: shows which other IDs this document is known as ², ³
see-also: references other relevant documents ²

RFCartography consists out of the IndexParser component, which parses the index file and makes its information available, and the RFCartography component, which builds the requested graphs and generates SVGs from them. This is held together by a Flask application that initially calls the IndexParser and provides the data returned by the parser to the RFCartographer, which it then repeatedly consults to generate graphs and render SVGs. Further, the Flask application implements the web frontend. The application is made available by a web server and a wsgi server.

When a request is received, the Flask application calls the RFCartographer to generate the subgraph belonging to the requested RFC. In order to do this, the initial RFC is added to a queue. Every RFC in the queue is added to the graph and all of its references are checked and added as edges to the graph. If a referenced document in not in the queue and hasn't been analyzed yet, it is added to the queue. To prevent long waiting times, a maximum depth can be specified. Once the queue is empty, the subgraph generation is completed and the graph can be returned.[°I profoundly apologise for this atrociously unreadable piece of code. In my defence: it seems to be working]

todo:      list[tuple[Document, int]]  = [(core, 0)]
done:      list[Document]              = []
graph:     MultiDiGraph                = MultiDiGraph()
graph.add_node(core)
nodes[core.type].append(core)

while len(todo) > 0:
    node: tuple[Document, int] = todo.pop(0)
    if node[0] not in done:
        done.append(node[0])
        if node[1] < max_depth or max_depth <= 0:
            for neighbor in node[0].get_references():
                if not neighbor[1].type in node_types:
                    continue
                if not graph.has_node(neighbor[1]):
                    graph.add_node(neighbor[1])
                    nodes[neighbor[1].type].append(neighbor[1])
                graph.add_edge(node[0], neighbor[1], reftype=neighbor[0])
                edges[neighbor[0]].append((node[0], neighbor[1]))
                todo.append((neighbor[1], node[1]+1))

Afterwards, the subgraph can be rendered into a SVG with the help of pyplot.

Of course this does not work without problems. First of all, RFCartography is really slow. Generating and rendering graphs takes a while, leading to long response times. This can be mitigated to a degree by caching responses, but to really improve on this issue, the application should probably use a database backend.[°I'll add this to RFCartography at some point]

Another issue seems to be due to a bug in pyplot. SVGs generated by RFCartography are supposed to have hyperlinks for each node that point to a page with details about that document. However, this does not work in some edge cases. If, for example, a subgraph consists out of only one node, pyplot does not add links to it. Further, it does not add links to the nodes if different shapes are used for different node types.[°Using different shapes would have been nice from an accessibility point of view, but since it doesn't work at the moment, RFCartography has to rely on colors only.]

You can try RFCartography yourself on rfcartography.undefinedbehavior.de. The source code can be found here. I'm not sure whether this project is actually useful for anyone, but it was fun to build it.

https://www.rfc-editor.org/rfc-index.xml ↩
https://www.rfc-editor.org/rfc-index.xsd ↩↩↩↩↩↩
J. Halpern, L. Daigle and O. Kolkman. (2016, May) RFC 7841: RFC Streams, Headers, and Boilerplates. https://www.rfc-editor.org/rfc/rfc7841 ↩↩↩↩↩

Slot Car Racing over IP (SCRoIP)

Error — Sun, 28 Aug 2022 16:05:00 +0200

When I was a child, I used to play a lot with my slot car racing track. After stumbling upon it again recently, I started wondering if I could build something to control it over the network. Not that I needed a remote control for it, but it seemed like a fun project. However, simply creating an API that offers precise control over the device seemed way too straight forward - the controlling mechanism itself should add its challenges. So I came up with the idea to control it with the network traffic itself. The more traffic is sent to the remote control, the faster the car is supposed to go.[°Use more bandwidth!!!]

In order to build the remote control, the first step was to figure out how the slot car racing system works. So I disassembled it and took a look at its wiring.

The underlying mechanism is fairly simple. The power supply provides about 14.8 V DC voltage, which is used to power the car's engine. The speed of the car is controlled by a variable resistor within the controller, which allows to control the voltage the car receives. Pushing the controller further down lowers the resistance, which raises the voltage available to the car and makes is go faster. A third wire is used to brake the car when the controller is in its neutral position by utilizing the current generated by the motor when the car is rolling out. A look at wikipedia confirms this analysis. ¹

Now that I understood the mechanism, the next step was to create a digital version of it, so that I could control the slot car racing track with a Raspberry Pi. My first idea was to simply use a digital potentiometer. However, searching for a suitable digipot, I quickly realized that there were no options with the desired dimensions of about 30 Ω available. Therefore I built it myself using a few resistors, solid state relais and multiplexers.[°I'm sure there are simpler solutions for this, but, well, it worked for me.]

For the sake of simplicity,[°or maybe out of laziness, who knows? :)] I did not add the braking mechanism from the analog controller to my digital version.

With the hardware fully assembled and soldered, it was time to have a look at the software. Python offers an easy way of implementing TCP servers.[°This project will use TCP. Even though the payload is immediately discarded, reliability is of the utmost importance and packet loss will not be accepted!!!] By default, it only supports IPv4, but IPv6 support can be added by setting address_family to AF_INET6.

if not legacy:
    self.address_family = AF_INET6
super(SCRoIPServer, self).__init__((address, port), SCRoIPRequestHandler)

The server's logic itself is implemented in the method handle() of the class SCRoIPRequestHandler. While there is an active connection, it reads from the socket and sums up the amount of data that was received. Once every 100 ms, it then calculates the average throughput for that time period and sets the raspi's GPIO pins accordingly.

while self.data != b'':
    try:
        self.data = self.request.recv(4096)
        self.traffic_counter += len(self.data)
    except TimeoutError:
        pass

    time = clock_gettime_ns(CLOCK_REALTIME)
    diff = time - self.last
    if (diff > 100000000):
        self.last = time
        traffic: float = self.traffic_counter / diff * 1000000000
        print(f"{traffic} Bytes/s")
        self.server.set_throttle(traffic)
        self.server.apply_throttle()
        self.traffic_counter = 0

The full server code can be found here.

Once the server was implemented as well, all that was left to do was to test it. So I deployed the server script to a Raspberry Pi, connected its GPIO pins to the controller and the controller to the slot car racing track. Using netcat, I was now able to accelerate the car with the network traffic I generated:

The video could not be loaded. You can try to download it directly here instead.

https://de.wikipedia.org/wiki/Autorennbahn#Fahrzeugsteuerung ↩