See ArchiveReorganisation for overall rationale.

Component reorganisation

Rationale

Following the upload permission changes described in ../Permissions, the distinction between the main and universe components is in many ways no longer as helpful as it used to be. It no longer corresponds to upload privileges in any firm way, and it does not form a superset of the packages installed by default for all Ubuntu flavours. main was originally intended to be a concise way to express what parts of Ubuntu were supported by Canonical. This model was conceived in a much simpler world - in fact, it predates the existence of an Ubuntu development community! - and it is showing its age.

Nomenclature

See ../Permissions for basic nomenclature.

A user's installation method typically implies initial interest in one or more package sets, and package management frontends should preferentially offer packages from those package sets where possible.

Package sets are exposed to users via package management, and thus care should be taken when rearranging them. While package set assignments are likely to be generated based on seeds in many or even most cases, the distinction allows us to shield users from rearrangements made purely for the convenience of developers.

The ogre model was conceived to allow Ubuntu to distribute the main component as a closed set, so that users could have a working system with no broken dependencies by including only main in /etc/apt/sources.list. This required ensuring that source packages in main could build-depend only on binary packages in main, and had the side-effect of ensuring that developers not permitted to upload to main cannot alter behaviour of packages in main (even accidentally) by changing the behaviour of their build-dependencies. See the "Permitted dependencies" column of the table at the top of this page for the current ogre-model structure.

Client-side changes

At minimum, it is necessary for build daemons to be able to build against a limited set of package sets. We believe that no syntactic extensions to APT's configuration files are necessary for this, but some semantic changes are necessary.

We will expose the package sets (zero or more) to which a package is assigned in a new Sets: field in the Packages file, managed by archive administrators using overrides. We then extend APT to permit /etc/apt/preferences configuration along the following lines:

Package: *
Pin: set core
Pin-Priority: 500

Package: *
Pin: set ubuntu-desktop
Pin-Priority: 500

Package: *
Pin: set *
Pin-Priority: -10

This would cause all packages in the "core" or "ubuntu-desktop" package sets to be installable as normal, and all other packages to be uninstallable.

FIXME: pinning on intersections of package sets, for ogre-model?

Mirroring tools need to be able to select by package set, similar to debmirror's existing --exclude-deb-section and --limit-priority options. This must be done strictly by omitting files from the mirror but retaining the corresponding entries in Packages, otherwise the mirror would need to be re-signed.

TODO: frontend search prioritisation

Ogre model (permitted dependencies and build-dependencies)

Any package set may be flagged as requiring ogre-model handling (typically, only package sets built from seeds will do this). Each such package set has a list of zero or more package sets on which it depends (similar to PPA dependencies).

Given a package in a set of package sets {A, B, C}, that package may only depend or build-depend on packages which are also in all those package sets, or a superset (e.g. {A, B, C, D}). Build daemons will enforce this rule for build-dependencies when building source packages.

Without this constraint, it would be impractical to build working partial mirrors, and many current source packages in the archive would break (since they rely on the current ogre model to install the appropriate provider for virtual packages). We believe that this design allows a lossless transition from the current archive layout.

Launchpad requirements

Launchpad needs at least the following extensions:

• A tool (perhaps edit-acl via the Launchpad API) needs to be able to manipulate the set of package sets to which a package is assigned.

• Package sets need to be written to the override indices, thereby causing new fields to be written to Packages files.
• Buildds need to be instructed on the set of package sets to use when building, per "Ogre model" above.

• The set of Ubuntu components as of the implementation of this specification should be changed to main, restricted-drivers, and restricted-software.

• The component and permission structures for Ubuntu releases prior to the implementation of this specification must be preserved.

Other nice-to-have extensions include:

• Implementation of pdiffs would improve download times substantially, particularly for people who do not want to point apt at the full set of available packages (who would otherwise need to download much larger Packages files in the new model).
• Launchpad Bugs should permit advanced queries by package set as well as by component.

We will prepare sample index files in advance, outside Launchpad, to allow people to experiment with tool changes and to get a feel for the resulting permissions.

Unresolved Issues

How does Archive Reorganisation affect langpack handling? Currently the distinction between "main" and "universe" is also used to determine how packages are translated.

BOF notes say that a "web-mirror-manager" specification will deal with partial mirroring; this does not appear to exist (but may not be necessary anyway).

It might be useful for a mirror to be able to provide a hint about which package sets it supports, to save round-trips for non-existent packages. (However, those extra round trips are likely to be due to a local mirror and thus less of a problem.) Alternatively, provide per-package-set Packages files (and signatures) so that mirrors can grab them?

More work needs to be done on the transition plan, especially for upgraders.

BOF Discussion

Design

Self-contained sets of packages have interest to sets of developers

Things to look at

• How can people know if a given package belongs to a given interest set
• Why does a given group of developers care about dependencies or build-dependencies?

MOTU

• we would no longer have a separate "universe" archive, but rather MOTU and universe would be defined by the set of unseeded packages
• what do we do with generalists who are shallow (not deep understanding, but interest in broad number of packages)?

Discussion

• Core are trusted to know what they're not qualified to touch
• MOTU are also trusted as well
• don't want to define a group by negative rights
• a package moved into a package set (eg Kubuntu) is somewhat like this
• Suppose: we have a group of "Ubuntu Developers" that can upload everywhere
  • package sets need to accept responsibility to collaborate