UDisks2

The new generation of UDisks2

Vratislav Podzimek

Red Hat Czech / Filesystems And Storage

<vpodzimeATredhat.com>

What UDisks Is

  • "the thing that lets you mount a USB flash drive without root permissions"
  • "the thing that lets you unlock your encrypted drive without root permissions"
  • a daemon providing a DBus API for monitoring and management of storage devices
  • running as root, using PolicyKit for authorization
  • udisksctl CLI
  • backend of GNOME Disks (Palimpsest)

A Brief History Of UDisks

Ancient Times and Middle Ages

  • original author: David Zeuthen (Red Hat)
  • original name: DeviceKit-disks
  • first commit: March 5 2008
  • renamed to UDisks: December 1 2009
  • bumped to UDisks2: February 16 2011
  • development until 2012/2013
  • maintenance until 2014
  • handed over to Martin Pitt (Debian/Ubuntu): 2014

Modern Times

  • conflict of interests: October 2014
    • Red Hat wanted "enterprise features" (e.g. LVM, for Cockpit)
    • David Zeuthen insisted on the idea of a "desktop/laptop storage daemon"
      "As I said a couple of months ago when we discussed this via email, I think this is a bad idea. Please move this to its own daemon/project or something; I don't want it in udisks." (bugs.freedesktop.org)
  • UDisks2 forked by Tomáš Bžatek (Red Hat)
    • the idea of an "enterprise-ready modular daemon"
    • implemented support for plugins/modules
    • first module: LVM2 (ports from Cockpit's storaged)

Post-Modern Times

  • parallel development (forked) and maintenance (origin)
  • March 6 2015: forked UDisks2 renamed to storaged (Peter Hatina, Red Hat)
    • new API org.storaged.Storaged
    • version reset to 2.0.0
  • a number of new modules added: iSCSI, zram, btrfs, bcache, LSM
  • January 28 2016: storaged as a drop-in replacement for UDisks2 (Fedora 25 Change)
    • back to the original API org.freedesktop.UDisks2
  • October 2016: storaged handed over to our team
  • February 15 2017: 
        Merge pull request #191 from martinpitt/the-grand-reunion

    Rename storaged back to udisks
    • Martin Pitt hired by Red Hat in the meantime

What we have been working on

  • wrote many, many DBus tests (150+)
  • transformed UDisks2 into a modern OSS project
    • TESTS
    • everything hosted on GitHub, doing pull requests, issues,...
    • Continuous Integration (Jenkins), Continuous Delivery (daily Copr builds)
    • versioned documentation
    • release early, release often (monthly)
    • fast and open responds at GitHub
  • participating in GSoC - Kai Lüke, GParted feature parity
  • ported UDisks2 to libblockdev 
        Merge pull request #260 from storaged-project/master-libblockdev

    Merge master-libblockdev to master

    76 files changed, 4940 insertions(+), 4851 deletions(-)
    [Total files:   928]
    [Total loc:   86694]

libblockdev

  • "...is a library for doing operations with block devices"
  • written in C, providing C and GObject Introspection API
  • plugin based
  • supports many technologies: partitions, LVM, MD RAID, LUKS, Btrfs, bcache, loop, file systems, swap, s390,...
  • "stateless", straightforward, providing simple high-level API
  • using libraries/DBus (where possible) or (mostly) parsing CLI output
  • benefits for UDisks2:
    • used by Blivet and the Anaconda OS installer
    • separating the daemon and DBus logic from the actual operations
    • heavily tested (293 tests ran on every change and every day)
    • simplicity

What we have been working on - cont.

Some numbers

Total commits: 2504
Total files: 928
Total loc: 86694
+-------------------------+-------+------+------+----------------+
| Author                  |   loc | coms | fils |  distribution  |
+=========================+=======+======+======+================+
| David Zeuthen           | 32588 | 1226 |   94 | 37.6/49.0/10.1 |
| Peter Hatina            | 27525 |  231 |  284 | 31.7/ 9.2/30.6 |
| Vojtech Trefny          |  7906 |  240 |   95 |  9.1/ 9.6/10.2 |
| Vratislav Podzimek      |  3685 |  274 |   97 |  4.3/10.9/10.5 |
| Gris Ge                 |  2781 |    6 |   25 |  3.2/ 0.2/ 2.7 |
| Tomas Bzatek            |  2692 |   43 |   30 |  3.1/ 1.7/ 3.2 |
| Martin Pitt             |  2175 |  208 |   74 |  2.5/ 8.3/ 8.0 |
| Tony Asleson            |  1460 |   62 |   30 |  1.7/ 2.5/ 3.2 |
| Marius Vollmer          |  1306 |   33 |   23 |  1.5/ 1.3/ 2.5 |
| Kai Lüke                |  1087 |    7 |    8 |  1.3/ 0.3/ 0.9 |

Reach outside of the Red Hat world

  • making new UDisks2 attractive
    • active development and frequent releases
    • being open to and greatful for third-party contributions
    • bug fixing (even old bugs from bugs.freedesktop.org)
    • making CI results publicly visible
  • new features:
    • file system and partition resize (Kai Lüke)
    • file system check/repair (Kai Lüke)
    • GetBlockDevices(), ResolveDevice() methods and Partitions property (Vojtěch Trefný)
    • generic DeviceOpen() method (Vojtěch Trefný)
    • Swapspace.SetLabel() method (Vojtěch Trefný)

Continuous Integration

  • Jenkins with 47 slaves (managed by ansible)
  • running tests on pull requests
  • running tests on the master branch

Endless Fighting

  • UDisks2 design
  • storage APIs (libraries)
  • stability of distributions
  • actually testing storage (as the only ones?!)
    • kernel panics
    • regressions in CLI/libraries
    • deadlocks in CLI/libraries
    • libraries that call abort(), wait(),...

Popcorn Time!

Thanks for your attention!

  • And big THANKS to all contributors to the UDisks project! In particular:
    • David Zeuthen
    • Martin Pitt
    • Tomáš Bžatek
    • Marius Vollmer
    • Peter Hatina
    • Tomáš Smetana
    • Vojtěch Trefný
    • Kai Lüke