Userspace-driven Kernel Object Configuration — The Linux Kernel documentation (2024)

What is configfs?¶

configfs is a ram-based filesystem that provides the converse ofsysfs’s functionality. Where sysfs is a filesystem-based view ofkernel objects, configfs is a filesystem-based manager of kernelobjects, or config_items.

With sysfs, an object is created in kernel (for example, when a deviceis discovered) and it is registered with sysfs. Its attributes thenappear in sysfs, allowing userspace to read the attributes viareaddir(3)/read(2). It may allow some attributes to be modified viawrite(2). The important point is that the object is created anddestroyed in kernel, the kernel controls the lifecycle of the sysfsrepresentation, and sysfs is merely a window on all this.

A configfs config_item is created via an explicit userspace operation:mkdir(2). It is destroyed via rmdir(2). The attributes appear atmkdir(2) time, and can be read or modified via read(2) and write(2).As with sysfs, readdir(3) queries the list of items and/or attributes.symlink(2) can be used to group items together. Unlike sysfs, thelifetime of the representation is completely driven by userspace. Thekernel modules backing the items must respond to this.

Both sysfs and configfs can and should exist together on the samesystem. One is not a replacement for the other.

Using configfs¶

configfs can be compiled as a module or into the kernel. You can accessit by doing:

mount -t configfs none /config

The configfs tree will be empty unless client modules are also loaded.These are modules that register their item types with configfs assubsystems. Once a client subsystem is loaded, it will appear as asubdirectory (or more than one) under /config. Like sysfs, theconfigfs tree is always there, whether mounted on /config or not.

An item is created via mkdir(2). The item’s attributes will alsoappear at this time. readdir(3) can determine what the attributes are,read(2) can query their default values, and write(2) can store newvalues. Don’t mix more than one attribute in one attribute file.

There are two types of configfs attributes:

• Normal attributes, which similar to sysfs attributes, are small ASCII textfiles, with a maximum size of one page (PAGE_SIZE, 4096 on i386). Preferablyonly one value per file should be used, and the same caveats from sysfs apply.Configfs expects write(2) to store the entire buffer at once. When writing tonormal configfs attributes, userspace processes should first read the entirefile, modify the portions they wish to change, and then write the entirebuffer back.

• Binary attributes, which are somewhat similar to sysfs binary attributes,but with a few slight changes to semantics. The PAGE_SIZE limitation does notapply, but the whole binary item must fit in single kernel vmalloc’ed buffer.The write(2) calls from user space are buffered, and the attributes’write_bin_attribute method will be invoked on the final close, therefore it isimperative for user-space to check the return code of close(2) in order toverify that the operation finished successfully.To avoid a malicious user OOMing the kernel, there’s a per-binary attributemaximum buffer value.

When an item needs to be destroyed, remove it with rmdir(2). Anitem cannot be destroyed if any other item has a link to it (viasymlink(2)). Links can be removed via unlink(2).

Configuring FakeNBD: an Example¶

Imagine there’s a Network Block Device (NBD) driver that allows you toaccess remote block devices. Call it FakeNBD. FakeNBD uses configfsfor its configuration. Obviously, there will be a nice program thatsysadmins use to configure FakeNBD, but somehow that program has to tellthe driver about it. Here’s where configfs comes in.

When the FakeNBD driver is loaded, it registers itself with configfs.readdir(3) sees this just fine:

# ls /configfakenbd

A fakenbd connection can be created with mkdir(2). The name isarbitrary, but likely the tool will make some use of the name. Perhapsit is a uuid or a disk name:

# mkdir /config/fakenbd/disk1# ls /config/fakenbd/disk1target device rw

The target attribute contains the IP address of the server FakeNBD willconnect to. The device attribute is the device on the server.Predictably, the rw attribute determines whether the connection isread-only or read-write:

# echo 10.0.0.1 > /config/fakenbd/disk1/target# echo /dev/sda1 > /config/fakenbd/disk1/device# echo 1 > /config/fakenbd/disk1/rw

That’s it. That’s all there is. Now the device is configured, via theshell no less.

Coding With configfs¶

Every object in configfs is a config_item. A config_item reflects anobject in the subsystem. It has attributes that match values on thatobject. configfs handles the filesystem representation of that objectand its attributes, allowing the subsystem to ignore all but thebasic show/store interaction.

Items are created and destroyed inside a config_group. A group is acollection of items that share the same attributes and operations.Items are created by mkdir(2) and removed by rmdir(2), but configfshandles that. The group has a set of operations to perform these tasks

A subsystem is the top level of a client module. During initialization,the client module registers the subsystem with configfs, the subsystemappears as a directory at the top of the configfs filesystem. Asubsystem is also a config_group, and can do everything a config_groupcan.

struct config_item¶

struct config_item { char *ci_name; char ci_namebuf[UOBJ_NAME_LEN]; struct kref ci_kref; struct list_head ci_entry; struct config_item *ci_parent; struct config_group *ci_group; struct config_item_type *ci_type; struct dentry *ci_dentry;};void config_item_init(struct config_item *);void config_item_init_type_name(struct config_item *, const char *name, struct config_item_type *type);struct config_item *config_item_get(struct config_item *);void config_item_put(struct config_item *);

Generally, struct config_item is embedded in a container structure, astructure that actually represents what the subsystem is doing. Theconfig_item portion of that structure is how the object interacts withconfigfs.

Whether statically defined in a source file or created by a parentconfig_group, a config_item must have one of the _init() functionscalled on it. This initializes the reference count and sets up theappropriate fields.

All users of a config_item should have a reference on it viaconfig_item_get(), and drop the reference when they are done viaconfig_item_put().

By itself, a config_item cannot do much more than appear in configfs.Usually a subsystem wants the item to display and/or store attributes,among other things. For that, it needs a type.

struct config_item_type¶

struct configfs_item_operations { void (*release)(struct config_item *); int (*allow_link)(struct config_item *src, struct config_item *target); void (*drop_link)(struct config_item *src, struct config_item *target);};struct config_item_type { struct module *ct_owner; struct configfs_item_operations *ct_item_ops; struct configfs_group_operations *ct_group_ops; struct configfs_attribute **ct_attrs; struct configfs_bin_attribute **ct_bin_attrs;};

The most basic function of a config_item_type is to define whatoperations can be performed on a config_item. All items that have beenallocated dynamically will need to provide the ct_item_ops->release()method. This method is called when the config_item’s reference countreaches zero.

struct configfs_attribute¶

struct configfs_attribute { char *ca_name; struct module *ca_owner; umode_t ca_mode; ssize_t (*show)(struct config_item *, char *); ssize_t (*store)(struct config_item *, const char *, size_t);};

When a config_item wants an attribute to appear as a file in the item’sconfigfs directory, it must define a configfs_attribute describing it.It then adds the attribute to the NULL-terminated arrayconfig_item_type->ct_attrs. When the item appears in configfs, theattribute file will appear with the configfs_attribute->ca_namefilename. configfs_attribute->ca_mode specifies the file permissions.

If an attribute is readable and provides a ->show method, that method willbe called whenever userspace asks for a read(2) on the attribute. If anattribute is writable and provides a ->store method, that method will becalled whenever userspace asks for a write(2) on the attribute.

struct configfs_bin_attribute¶

struct configfs_bin_attribute { struct configfs_attribute cb_attr; void *cb_private; size_t cb_max_size;};

The binary attribute is used when the one needs to use binary blob toappear as the contents of a file in the item’s configfs directory.To do so add the binary attribute to the NULL-terminated arrayconfig_item_type->ct_bin_attrs, and the item appears in configfs, theattribute file will appear with the configfs_bin_attribute->cb_attr.ca_namefilename. configfs_bin_attribute->cb_attr.ca_mode specifies the filepermissions.The cb_private member is provided for use by the driver, while thecb_max_size member specifies the maximum amount of vmalloc bufferto be used.

If binary attribute is readable and the config_item provides act_item_ops->read_bin_attribute() method, that method will be calledwhenever userspace asks for a read(2) on the attribute. The conversewill happen for write(2). The reads/writes are buffered so only asingle read/write will occur; the attributes’ need not concern itselfwith it.

struct config_group¶

A config_item cannot live in a vacuum. The only way one can be createdis via mkdir(2) on a config_group. This will trigger creation of achild item:

struct config_group { struct config_item cg_item; struct list_head cg_children; struct configfs_subsystem *cg_subsys; struct list_head default_groups; struct list_head group_entry;};void config_group_init(struct config_group *group);void config_group_init_type_name(struct config_group *group, const char *name, struct config_item_type *type);

The config_group structure contains a config_item. Properly configuringthat item means that a group can behave as an item in its own right.However, it can do more: it can create child items or groups. This isaccomplished via the group operations specified on the group’sconfig_item_type:

struct configfs_group_operations { struct config_item *(*make_item)(struct config_group *group, const char *name); struct config_group *(*make_group)(struct config_group *group, const char *name); void (*disconnect_notify)(struct config_group *group, struct config_item *item); void (*drop_item)(struct config_group *group, struct config_item *item);};

A group creates child items by providing thect_group_ops->make_item() method. If provided, this method is called frommkdir(2) in the group’s directory. The subsystem allocates a newconfig_item (or more likely, its container structure), initializes it,and returns it to configfs. Configfs will then populate the filesystemtree to reflect the new item.

If the subsystem wants the child to be a group itself, the subsystemprovides ct_group_ops->make_group(). Everything else behaves the same,using the group _init() functions on the group.

Finally, when userspace calls rmdir(2) on the item or group,ct_group_ops->drop_item() is called. As a config_group is also aconfig_item, it is not necessary for a separate drop_group() method.The subsystem must config_item_put() the reference that was initializedupon item allocation. If a subsystem has no work to do, it may omitthe ct_group_ops->drop_item() method, and configfs will callconfig_item_put() on the item on behalf of the subsystem.

Important:

drop_item() is void, and as such cannot fail. When rmdir(2)is called, configfs WILL remove the item from the filesystem tree(assuming that it has no children to keep it busy). The subsystem isresponsible for responding to this. If the subsystem has references tothe item in other threads, the memory is safe. It may take some timefor the item to actually disappear from the subsystem’s usage. But itis gone from configfs.

When drop_item() is called, the item’s linkage has already been torndown. It no longer has a reference on its parent and has no place inthe item hierarchy. If a client needs to do some cleanup before thisteardown happens, the subsystem can implement thect_group_ops->disconnect_notify() method. The method is called afterconfigfs has removed the item from the filesystem view but before theitem is removed from its parent group. Like drop_item(),disconnect_notify() is void and cannot fail. Client subsystems shouldnot drop any references here, as they still must do it in drop_item().

A config_group cannot be removed while it still has child items. Thisis implemented in the configfs rmdir(2) code. ->drop_item() will not becalled, as the item has not been dropped. rmdir(2) will fail, as thedirectory is not empty.

struct configfs_subsystem¶

A subsystem must register itself, usually at module_init time. Thistells configfs to make the subsystem appear in the file tree:

struct configfs_subsystem { struct config_group su_group; struct mutex su_mutex;};int configfs_register_subsystem(struct configfs_subsystem *subsys);void configfs_unregister_subsystem(struct configfs_subsystem *subsys);

A subsystem consists of a toplevel config_group and a mutex.The group is where child config_items are created. For a subsystem,this group is usually defined statically. Before callingconfigfs_register_subsystem(), the subsystem must have initialized thegroup via the usual group _init() functions, and it must also haveinitialized the mutex.

When the register call returns, the subsystem is live, and itwill be visible via configfs. At that point, mkdir(2) can be called andthe subsystem must be ready for it.

An Example¶

The best example of these basic concepts is the simple_childrensubsystem/group and the simple_child item insamples/configfs/configfs_sample.c. It shows a trivial object displayingand storing an attribute, and a simple group creating and destroyingthese children.

Hierarchy Navigation and the Subsystem Mutex¶

There is an extra bonus that configfs provides. The config_groups andconfig_items are arranged in a hierarchy due to the fact that theyappear in a filesystem. A subsystem is NEVER to touch the filesystemparts, but the subsystem might be interested in this hierarchy. Forthis reason, the hierarchy is mirrored via the config_group->cg_childrenand config_item->ci_parent structure members.

A subsystem can navigate the cg_children list and the ci_parent pointerto see the tree created by the subsystem. This can race with configfs’management of the hierarchy, so configfs uses the subsystem mutex toprotect modifications. Whenever a subsystem wants to navigate thehierarchy, it must do so under the protection of the subsystemmutex.

A subsystem will be prevented from acquiring the mutex while a newlyallocated item has not been linked into this hierarchy. Similarly, itwill not be able to acquire the mutex while a dropping item has notyet been unlinked. This means that an item’s ci_parent pointer willnever be NULL while the item is in configfs, and that an item will onlybe in its parent’s cg_children list for the same duration. This allowsa subsystem to trust ci_parent and cg_children while they hold themutex.

Item Aggregation Via symlink(2)¶

configfs provides a simple group via the group->item parent/childrelationship. Often, however, a larger environment requires aggregationoutside of the parent/child connection. This is implemented viasymlink(2).

A config_item may provide the ct_item_ops->allow_link() andct_item_ops->drop_link() methods. If the ->allow_link() method exists,symlink(2) may be called with the config_item as the source of the link.These links are only allowed between configfs config_items. Anysymlink(2) attempt outside the configfs filesystem will be denied.

When symlink(2) is called, the source config_item’s ->allow_link()method is called with itself and a target item. If the source itemallows linking to target item, it returns 0. A source item may wish toreject a link if it only wants links to a certain type of object (say,in its own subsystem).

When unlink(2) is called on the symbolic link, the source item isnotified via the ->drop_link() method. Like the ->drop_item() method,this is a void function and cannot return failure. The subsystem isresponsible for responding to the change.

A config_item cannot be removed while it links to any other item, norcan it be removed while an item links to it. Dangling symlinks are notallowed in configfs.

Automatically Created Subgroups¶

A new config_group may want to have two types of child config_items.While this could be codified by magic names in ->make_item(), it is muchmore explicit to have a method whereby userspace sees this divergence.

Rather than have a group where some items behave differently thanothers, configfs provides a method whereby one or many subgroups areautomatically created inside the parent at its creation. Thus,mkdir(“parent”) results in “parent”, “parent/subgroup1”, up through“parent/subgroupN”. Items of type 1 can now be created in“parent/subgroup1”, and items of type N can be created in“parent/subgroupN”.

These automatic subgroups, or default groups, do not preclude otherchildren of the parent group. If ct_group_ops->make_group() exists,other child groups can be created on the parent group directly.

A configfs subsystem specifies default groups by adding them using theconfigfs_add_default_group() function to the parent config_groupstructure. Each added group is populated in the configfs tree at the sametime as the parent group. Similarly, they are removed at the same timeas the parent. No extra notification is provided. When a ->drop_item()method call notifies the subsystem the parent group is going away, italso means every default group child associated with that parent group.

As a consequence of this, default groups cannot be removed directly viarmdir(2). They also are not considered when rmdir(2) on the parentgroup is checking for children.

Dependent Subsystems¶

Sometimes other drivers depend on particular configfs items. Forexample, ocfs2 mounts depend on a heartbeat region item. If thatregion item is removed with rmdir(2), the ocfs2 mount must BUG or goreadonly. Not happy.

configfs provides two additional API calls: configfs_depend_item() andconfigfs_undepend_item(). A client driver can callconfigfs_depend_item() on an existing item to tell configfs that it isdepended on. configfs will then return -EBUSY from rmdir(2) for thatitem. When the item is no longer depended on, the client driver callsconfigfs_undepend_item() on it.

These API cannot be called underneath any configfs callbacks, asthey will conflict. They can block and allocate. A client driverprobably shouldn’t calling them of its own gumption. Rather it shouldbe providing an API that external subsystems call.

How does this work? Imagine the ocfs2 mount process. When it mounts,it asks for a heartbeat region item. This is done via a call into theheartbeat code. Inside the heartbeat code, the region item is lookedup. Here, the heartbeat code calls configfs_depend_item(). If itsucceeds, then heartbeat knows the region is safe to give to ocfs2.If it fails, it was being torn down anyway, and heartbeat can gracefullypass up an error.