transitioning GNU Linux-libre to git-based releases

Thu May 7 12:59:08 UTC 2020

Richard Stallman suggested us to switch to a git repo as the primary
means for us to release cleaned-up sources and cleaning-up scripts.
What follows is a summary of the main points and upcoming changes we
talked about, with a few questions for the community on other changes we
could make in the process.

# Repo structure

What I have in mind is a structure like that of the repo maintained by
Jason Self at <https://jxself.org/git/linux-libre.git>, namely:

- deblobbing scripts get branches and tags mirroring those in the SVN
  repo

- cleaned up source trees are each an initial commit and a signed tag,
  no branches for those.

Rationale for this unusual structure is that, should we find a
deblobbing error in some old release, that lets non-Free Software in it,
it's easy to withdraw it, without affecting any subsequent releases;
branch histories would make subsequent releases refer back to the older
release and retain the non-Free Software in their history.

# Release process

Instead of putting out tarballs that would later be imported into this
tree, I'd make the importing part of the release process.  They'd likely
be available much faster, without waiting for tarball compression and
signing to complete.

Instead of the current deblob-main script, we'd use alternate logic to
drive the cleaning up and packaging, including the generation of diffs
and tarballs with git.

I'll probably no longer publish a git-based deblob-main along with each
release; it will likely be maintained and published separately, if at
all.  I still can't tell how sensible it would be to publish the
git-based replacement(s?) for deblob-main, but I've learned that
deblob-main changes very little and yet having it as part of release
series was a bit of a hindrance in improving the release process in the
long term.  Plus, the git-based process will take some experimentation
before it settles down, probably with manual processes at first, and
nothing quite as elaborate as the logic to partially modify tarballs as
in deblob-main, so...  don't be surprised if deblob-main is gone without
anything to take its place, at least for a while.

# Tarballs

We'd keep on creating and publishing tarballs, at least for now.  I'm
not sure whether it makes sense to create all of .bz2, .xz and .lz,
though.  Please let me know whether you rely on any of these formats;
we'd keep them mainly for backward-compatibility, since any of them
could/would from now on be created from the git repo with git archive.

## Dir Prefix

In order to minimize the differences between our tarballs and
upstream's, we've historically released e.g. linux-libre-x.y-gnu.tar
exploding to linux-x.y/, like the corresponding upstream tarball.

This transition would offer a good opportunity to fix this unusual
behavior, and make upcoming tarballs explodes to linux-libre-x.y.gnu/.
However, for backward compatibility, it seems to me that it would be
less disruptive to keep the current unusual behavior.

The idea being that those who make a change because of this change of
ours might as well switch to the git release repo, creating archives
with any prefix they like, while those who wish to stick to tarballs
don't need to change anything.

## Tar Metadata

Our single-initial-commit will likely generate much different metadata
(timestamps, user/group ownership) than what's obtained from git archive
for corresponding upstream releases.

Though it might be possible to try to duplicate the metadata that goes
in upstream tarballs, I don't see reason to try to emulate this artifact
of our previous release procedure, that served mainly to reduce xdeltas
(see below).  Please let me know if you find otherwise.

# Binary deltas

We have historically published binary deltas from upstream tarballs to
our own, so as to enable ours to be reconstructed even after we remove
them from our release archive.  xdelta v1 had pretty small diffs, but
that version has not been maintained for a very long time, and xdelta v2
and v3 haven't done such a good job IMHO.

With a git repo holding all releases, the reason for the xdeltas ceases
to exist: the incremental disk space requirements for each release in
git are negligible, compared with entire tarballs, so we can afford to
keep them all forever.  I'm therefore inclined to simply discontinue the
creation of xdeltas.

If there is interest, however, I could still create them for the time
being.  Please let me know if you'd still find them useful, being able
to reconstruct our tarballs with git archive.

# Signing old releases

I intend to start from Jason's repo.  He's imported our entire release
history there, and I trust the data in there to reflect that accurately.
However, the tags are not signed by the primary Linux-libre key.

I wonder if there'd be any point in (comparing? and) signing each of the
tags for the old releases with the primary Linux-libre key, so that we
can entirely obviate the current release archive.  That doesn't mean
it's going away any time soon, just that it becomes entirely redundant.

In the process, I might generate and publish xdeltas between git-archive
tarballs generated from the newly-verified and signed tags, and
corresponding upstream tarballs, or our own previously-released
tarballs, as a repackaging of earlier releases.

Would that re-signing and repackaging be of any use for anyone?

Please let us know of any other concerns related with these changes that
you might have.  I hope to make the transition shortly, certainly before
5.7, maybe as soon as the next batch of stable updates.

-- 
Alexandre Oliva, freedom fighter    he/him    https://FSFLA.org/blogs/lxo/
Free Software Evangelist              Stallman was right, but he's left :(
GNU Toolchain Engineer           Live long and free, and prosper ethically
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 818 bytes
Desc: not available
URL: <http://www.fsfla.org/pipermail/linux-libre/attachments/20200507/4436faa8/attachment.sig>