Linux-libre architecture and how to modify it for other uses cases?

Thu Oct 14 00:11:40 UTC 2021

Hi,

I'm currently working on porting Replicant to Android 11, and while
doing that we decided to base our work on top of an upstream kernel,
and I also got some funding from NLnet to do all that.

Since the difference with upstream Linux is now much smaller than with
the vendor kernels, and that we don't want to and can't redistribute
nonfree software, we need to remove any nonfree software found in
Android[1] or in Linux.

And here I thought that it was a good idea to take a proactive approach
and actually try to find nonfree software and remove it, instead of
waiting to find it accidentally and removing them when it happens.

And since linux-libre exists and does that (among other things), I
thought that it would be a good idea to try to reuse it in one way or
another.

However even if Replicant doesn't have any instructions to install
nonfree firmwares and that their use is strongly discouraged, including
with work to make it easier not to use them, by integrating
support for ath9k_htc compatible external WiFi dongles, or by
shipping scripts to disable the modem, we are not yet in a situation
where we can completely block the use of nonfree firmwares.

As far as I know there are only two smartphones (The Openmoko
Freerunner and the Librem5) where the internal WiFi works without
requiring distributions to ship nonfree firmwares (this is because the
nonfree firmwares are integrated in a dedicated memory connected
directly to the WiFi chip).

And Replicant can't support the Freerunner (not enough RAM) and doesn't
support the Librem5 yet (we want to release Replicant 11 before adding
new devices).

In contrast it is possible to find laptops where either the internal
WiFi works out of the box (like RYF compliant laptops) or
where it is possible to install internal WiFi cards that work with
linux-libre (like x86 Libreboot compatible laptops).

Beside not blocking nonfree firmwares, we also need to have patches on
top of linux-libre but since linux-libre has source code in git, that
is not an issue[2] as we can simply add our patches on top of the
linux-libre commit[3] and make it easy for anybody to trust the
linux-libre tag and only review the changes we have on top.

Given the nature of the changes we want to do I also thought that it
was a good idea to rebrand the kernel source. To do that I modified the
deblob script to add information about the Replicant kernel that
explains the (planned) differences with linux-libre, and that worked
well. 

It was also easy to unify the scripts and the sources tag of a given
version by moving the scripts with git mv, and merging both with git
merge. I also made a Makefile.deblob (that I attached) that downloads
Linux and run the deblob scripts to produce the modified version of
linux-libre. 

The idea is to then commit the script changes and the resulting
linux-libre source code changes in the same commit(s) to make the
code review(s) easier and as small as possible.

The part that I didn't manage to do yet and that looks complicated for
me is to find a way for Replicant to reuse the deblobing part without
reusing the firmware blocking part in a way that requires the least
amount of maintenance. 

The idea here is to enable people not familiar with linux-libre to very
easily rebase the Replicant changes on top of more recent kernels
versions, like we do with Linux, but while still keeping deblobed
kernels.

To implement the non-removal of nonfree firmwares, I tried to do the
least invasive changes possible in the deblob-5.10 script from
scripts/v5.10.72-gnu1 by emptying the functions that patches Linux to
reject the nonfree firmwares (so reject_firmware,
reject_firmware_nowarn, maybe_reject_firmware, and undefine_macro).

reject_firmware looked like that for instance:
> reject_firmware () {
>   return 0
> }
and the content of the others functions mentioned above were also
changed "return 0".

But then I got lot of /* DEBLOBED */ that would popup in the middle of
(firmware loading?) function calls like that (I don't recall exactly how
they looked):
> sta/* DEBLOBED */re(&foo, &bar);.

So I wonder if what I was doing here is the best approach.

In general I'm also having a hard time understanding the overall
architecture without spending a lot of going deep into the code.

I guess the code is complex because the problem it tries to solve is
far from trivial (automatically detecting nonfree firmwares looks really
impressive to me) and that it needs regular expressions anyway.

What I didn't understand well is what deblob-check is supposed to do
beside finding binary code. 

Is it supposed to actually patch the code as well? 

Does it handle the various backends (like sed, python, perl, awk) in a
generic way? 

Are the regex generic? Or are some primitive reimplemented in the
various backends? 

And how does deblob-5.10 and deblob-check are supposed to interact
together? I found where but I didn't understand exactly what part was
left to what script and if there was some sort of communication between
both or not.

Also each run of the scripts takes a very long time, so doing complete
tests would take quite some time. 

So far I only took the following (naive) approach (which is implemented
by the Makefile.deblob I attached):
- I would be in a git revision that is based on sources/v5.10.72-gnu1
  with also the deblob scripts merged in (potentially with my changes
  on top).
- I would delete the linux-libre source code and then unpack the
  linux source code
- I would then run the deblob-5.10 script, which would take way too
  long to complete.

Are there faster setups?
Could some of the faster setups suit my needs? 

Or would it be possible for me to somehow parallelize the deblob
scripts without too much efforts?

References:
-----------
[1]For Android, basing Replicant on the official Android source code
   instead of LineageOS makes that easier.
[2]Android doesn't have package definitions, instead all the source
   code is assembled from many git repositories in a single tree
   structure. Because of that everything needs to be in git
   repositories.
[3]For example we can simply cherry-pick our patches on top of the
   sources/v5.10.72-gnu1 tag of the linux-libre git repository[4].
[4]git://linux-libre.fsfla.org/releases.git

Denis.
-------------- next part --------------
A non-text attachment was scrubbed...
Name: Makefile.deblob
Type: application/octet-stream
Size: 2315 bytes
Desc: not available
URL: <http://www.fsfla.org/pipermail/linux-libre/attachments/20211014/2dee280a/attachment.obj>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 833 bytes
Desc: OpenPGP digital signature
URL: <http://www.fsfla.org/pipermail/linux-libre/attachments/20211014/2dee280a/attachment.sig>