Disposable implementation

Warning

This page is intended for advanced users.

Disposable behavior

A disposable template is not a disposable qube in itself, but a qube that can be used to create different disposable types: normal disposables and named disposables. This intermediary template serves different functions: first, it allows customization of the private volume of a disposable, and second, it provides a degree of inheritance that would not be possible with normal templates. It has the template_for_dispvms property enabled, being a DVMTemplateMixin.

A disposable is a qube with the DispVM class and is based on a disposable template. Every disposable type has all of its volumes configured to disable save_on_stop, therefore no changes are saved on shutdown. Normal disposables enable the property auto_cleanup by default, thus Qubes OS automatically removes the qube upon shutdown. Named disposables don’t enable auto_cleanup by default, thus the qube skeleton is not removed upon shutdown, thus allowing to keep qube settings.

Named disposables are useful for service qubes, as referencing static names is easier when the qube name is mentioned on Qrexec policies (qubes.UpdatesProxy target) or as a property of another qube, such as a disposable net qube which is referenced by downstream clients in the netvm property.

Disposables are generally named according to the disp1234 scheme, where 1234 is derived from the dispid property, a random integer ranging from 0 to 9999 with a fail-safe mechanism to avoid reusing the same value in a short period. Named disposables have a user-set permanent name.

Creating disposables through Qrexec

Every qube has the default_dispvm property, which defines which disposable template will be used to spawn disposables for this qube by default (when using actions such as “open in disposable”). It can have one of three values:

  • a disposable template

  • empty value

  • default system value (this will use the system-wide default_dispvm property)

In case of disposables, this property is by default set to their own disposable template, to avoid data leaks through, for example, using unintended network access paths.

This value is also used in case of Qrexec policy: when policy rules use the @dispvm tag, it translates to “a disposable based on the source qube’s default_dispvm. It is most commonly used to open files and URLs, (qubes.OpenInVM and qubes.OpenURL, respectively).

If you want to allow creating disposables based on different disposable templates, you can use the disposable template name or tag as destination. In particular:

  • @dispvm:DISPOSABLE_TEMPLATE, where DISPOSABLE_TEMPLATE is the desired template;

  • @dispvm:@tag:CUSTOM_TAG, where CUSTOM_TAG is the tag of your choice.

Preloaded disposables

Starting new disposables can be a slow process. To speed it up, Qubes OS provides disposable preloading, which enables fast retrieval of fresh disposables.

Instead of performing the whole startup each time a new disposable is requested, preloading keeps prepared (running and paused) qubes in the background, ready to be used when needed. In order to accomplish this, they are started in the background without user interaction, hidden from most graphical applications by being internal. They are unpaused (transparently) when a disposable qube is requested by the user.

Preloaded disposable’s stages

There are several stages a disposable goes through when preloading is enabled:

  1. Preload: The qube is created and marked as preloaded. The qube is not visible in GUI applications.

  1. Startup: Begins qube startup, starts basic services and attempts to pause the qube.

  2. Request: Triggered by the user requesting a new disposable qube. The qube is removed from the preload list. If qube startup has not yet reached pause, the pause stage is skipped.

  1. Used: The qube is marked as used and will be unpaused/resumed (if needed). At this stage, GUI applications treat the qube as any other disposable and the qube object is returned to the caller if requested.

Preloaded disposable’s worry-free life-cycle

There are several events that may trigger the creation or deletion of preloaded disposables. Qubes OS attempts to maintain as many preloaded disposables as the user set through the preload-dispvm-max feature, and any preloaded disposables that are invalid will be deleted or replaced as soon as possible. This is balanced by the need to ensure the system is responsive, thus the refreshing of disposables may take some time.

Preloaded disposable’s management

Maximum number of preloaded disposables is defined by the preload-dispvm-max feature. This feature can be set on any disposable template (this defines the maximum number of disposables preloaded from this template), and on the dom0 (this defines the maximum number of disposables preloaded from the system-wide default disposable template). In case those values conflict, the dom0 value takes precedence (for example, if preload-dispvm-max is set to 0 on the default-dvm disposable template, which is the default disposable template, and to 4 on dom0, Qubes OS will attempt to preload 4 disposables from default-dvm).

These are common events that trigger changes in preloaded disposables quantity:

  • Create or remove:

    • Changing the preload-dispvm-max feature;

    • Changing system-wide default_dispvm if system-wide preload-dispvm-max is set to a different value than the disposable template setting;

  • Refill:

    • (Re)starting qubes-preload-dispvm.service;

    • Requesting a disposable;

    • Using a preloaded disposable (both this and requesting a disposable trigger a refill, in case one of them fails - this ensures maximum preloading if possible);

  • Refresh (delete old preloaded disposables and create then anew):

    • Updating the volumes of a template or disposable template;

    • Qubesd was interrupted mid preload creation, on the next service restart, domain-load of the disposable template.

Availability gaps

Some events that may cause a temporary gap in preloaded disposable availability:

  • There is not enough memory to preload at the moment;

  • Service to check if the system is fully operational has failed and will remove the qube. This will not cause repeated attempts as it most likely would lead to the same outcome to infinity.

Bootstrapping Preloaded disposables

To bootstrap the creation of preloaded disposables after boot, the service qubes-preload-dispvm.service is used instead of domain-load of the disposable template because it relies on systemd to:

  • Order this action after the autostart or standard qubes - they must precede in order to have a functional system;

  • Skip preloading if kernel command line prevents autostart.

Preloaded disposables’ memory management

Qubes OS attempts to manage a preloaded disposable’s qube memory directly before the pause stage, as it is impossible to negotiate memory with a paused qube. At the domain-pre-paused stage, memory is set to use preferred memory value. In qmemman terms, preferred memory is just enough to have the qube running, and so, hopefully as much as possible of qube memory is retrieved to be used by other, currently running qubes.

This process can take a bit of time because it depends on how fast the qube can free up memory. In qubes.qmemman.systemstate, there is a timeout (CHECK_PERIOD) and a threshold in transfer speed (CHECK_MB_S) when attempting to balloon. When both of these conditions are met, the memory management is done and the preloaded disposable can be paused.

We do not worry about the time this process takes, even though it does prolong the preload creation stage. In the long run, the memory usage is more important. Unfortunately, qmemman is synchronous, so only one request can be made at a time, thus anything that takes too much time on qmemman could prevent balancing memory usage of other qubes.

Preloaded disposable’s pause

Preloaded disposables are paused because:

  • it makes detecting if a qube was used without being requested with qubes.vm.dispvm.DispVM.from_appvm(). Not every communication with a qube goes through qubesd, it may happen via Qrexec or GUI daemon;

  • CPU scheduling economy: a paused domain is not eligible for it;

  • Cronjob, timers and other things that don’t block the init system and service manager completion won’t run and alter a clean state.

But this comes at a cost:

  • Memory management before pause may take some seconds. As described above, this does not increase time to use the preloaded qube but can slow down the system as a whole, as qmemman can not balance memory usage of other qubes in the mean time.

Preloaded disposable’s security

As preloaded disposables are started before being used, some measures to prevent accidental tampering and reuse were introduced:

  • The qube has the internal feature enabled. Qubes GUI applications hide internal qubes;

  • When requesting a disposable, the qube object is only returned to the user once it has finished preloading;

  • The qube is paused as the last stage of preloading. This means the domain-unpaused event can be used to mark the qube as used and remove it from the preload list, avoiding reliabce on qubes.vm.dispvm.DispVM.from_appvm();

  • The GUID and Audio daemon only connect to the GUI agent and audio agent in the qube after the preloaded disposable is marked as used. This prevents autostarted applications from appearing on the screen before they are ready or before pause, which could be confusing. Enabling a GUI is controlled by the is_preload property - when it is set to False, GUI and audio connections are initiated.

To ensure reliability:

  • The preload-dispvm-threshold feature controls how much free memory must be present on the system before attempting to create a new preloaded disposable. Used to ensure preloaded disposables do not consume all available memory, which would prevent starting other qubes.

To have late GUI daemon but an early GUI agent, changes have been made that limit the usability on sys-gui. Events such as plugging or removing external monitors can’t work, it will be ignored by Xephyr.

Alternatives considered

For an alternative to be considered for implementation, it must meet the following requirements:

  • No memory or vcpus restrictions such as limiting to a few number of vcpus or assigns memory on request (can be slow).

  • Performant as much as a normal disposable even on long running sessions;

  • Caller transparency, no change necessary for callers, the request must be transparent and the server must find the fastest option. This is to avoid transition burden (API breakage).

From the evaluated options, only preload queue meets all requirements.

Restoration from savefile

Description: Disposable template booted, its image was dumped (suspend to disk), newly disposables would restore this image to become their own.

Evaluation:

  • Used in R3.2, worked at that time, when there was only one disposable template available, see next points of why it can’t be used anymore.

  • Incompatible with multiple vcpus.

  • Some memory issues.

  • Savefile creation takes a long time. The disposable qube savefile contains references to template rootfs and CoW files, if there is a modification on the template or disposable template, it took longer than 2.5 minutes to generate the next disposable.

Xen domain fork

Description: domain forking is the process of creating a domain with an empty memory space and a parent domain specified from which to populate the memory when necessary. For the new domain to be functional the domain state is copied over as part of the fork operation (HVM params, heap allocation etc). This description was sourced from [Xen-devel] [RFC PATCH for-next 17/18] xen/mem_sharing: VM forking, Tamas K Lengyel.

Evaluation:

  • Shares too much information from the trunk to the forks. This appears to have improved if not totally fixed on Linux 6.14, as mentioned by Andrew Cooper on Qubes OS Summit 2025;

  • Requires changing properties after the fork is done, this includes, but not limited to, xid of connected qubes, network uplink;

  • Not designed for long running sessions, the initial intention was fuzzing. As fast as the creation can be, the usage may be slower as memory is mapped on request. Xen doesn’t have a poper CoW support for domain memory, so making a full copy of a domain on fork also has some overhead;

  • Tamas K Lengyiel VM Forking & Hypervisor-Based Fuzzing with Xen presentation during the Open Source Summit Europe in 2022, showed impressive results on CPU i5-8350U, an average of time of 0.745 ms per fork (created 1300 VM/s). These fast results were later explained that was due to not initializing the whole VM memory on the fork unless it was requested, as explained on the point above. Still impressive results but current usage is limited to fuzzing.

Preload queue

Description: Start disposables and queue them in a disposable template feature, requested disposables will prefer to retrieve disposables from this list.

Evaluation:

  • Because the qube is running prior to being requested, multiple components have to be patched to support it to various levels off difficulty. Excluding from backups to allowing removal of disposable templates that only have preloaded disposables to even stranger issues such as deferring net qube change from a preloaded disposable where the old net qube has already been purged from the system.

  • The biggest difference between the queue and the other alternatives is that this solution works, is reliable and fulfills all requirements. A proper solution would be patching upstream Xen to implement CoW, but that would involve a lot more work than what the Qubes Team can provide with current resources.