MAAS controller communication

by Bill Wear on 11 March 2020

Much of the functionality of MAAS is contained in controllers. There are two basic types: a region controller and a rack controller. It’s useful to pull back and take a quick look at how these controllers work and interact. This will help you get a better picture of how MAAS operates.

Region controller

The region controller is responsible for a data centre or, at least, a single region within that data centre. It deals with operator requests, managing user interaction through some of the more abstract interfaces, such as:

• REST API server (TCP port 5240)
• DNS
• caching HTTP proxy (such as squid cache-and-forward proxy)
• web UI

When you make a request via the MAAS Web UI, you’re actually initiating an API call to the region controller. Multiple fabrics are used by MAAS to accommodate subdivisions within a single region, such as multiple floors in a data centre — where a fabric simply provides a mechanism for a collection of individual VLANs to communicate with one another.

Rack controllers

In the MAAS model, one or more rack controllers provide high-bandwidth services to machines. The rack controllers directly manage the bare metal. By contrast with the region controller, the rack controller communicates in lower-level protocols like:

• DHCP
• TFTP
• HTTP (for images)
• power management (BMC/IPMI)

A rack controller is attached to each fabric. As the name implies, a typical setup is to have a rack controller in each data centre server rack. The rack controller will cache large items for performance, such as operating system install images, but maintains no independent state other than the credentials required to talk to the region controller.

PostgreSQL database

Both the region and rack controllers are stateless with respect to machines, that is, neither of them store any details of a particular bare-metal server that they may be responsible for controlling. Instead, this information is stored in a PostgreSQL database that is managed and accessed by the region controller. Although the region controller is managing asynchronous events, it contains separate logic to manage the database synchronously, keeping the machine data fresh and accurate.

Controller interaction

The MAAS region and rack controllers interact in a number of different ways, depending upon the operation you’ve requested. By way of an example, we’ll consider the process of commissioning a machine. When MAAS commissions a machine, it essentially takes ownership of the unit, gathering information on its available resources (CPU, RAM, storage, and so forth).

Here’s a rough idea of what that sequence looks like – a sequence that is representative of the communication between controllers:

1. An operator makes a request of MAAS, either via the Web UI or the API. In this case, we’ll assume the user requested that the machine in question be commissioned.
2. MAAS translates this to an API request to the region controller.
3. The region controller locates the rack controller that has BMC access to the machine in question, that is, the rack controller that can power on that machine.
4. That same rack controller powers on the machine via IPMI request.
5. The rack controller tasked with providing DHCP handles assigning an IP address to the machine via the DORA sequence (Discover, Offer, Request, Acknowledge). Note that this rack controller doesn’t have to be the same one that powers on the machine.
6. The DHCP-managing rack controller inserts itself as the DHCP “next-server” and requests a network boot.
7. (Still) the same rack controller RPCs the region controller to get what’s needed to boot an ephemeral Ubuntu kernel, namely the kernel, any kernel parameters, an initrd daemon, and a squashfs load.
8. That same rack controller transforms the RPC response from the region controller into a valid PXE config and tells the machine to come get its files.
9. The booting machine loads the kernel and initrd, boots with that initrd, and then loads the squashfs, eventually making its way up to an ephemeral Ubuntu instance.
10. The booted machine pulls cloud-init metadata from the region controller, proxying through the rack controller.
11. cloud-init uses this metadata to gather resource information about the machine and pass it back to the region controller, again proxied by the rack controller.
12. The region controller stores this machine information in a postgres database that is accessible only to the region controller, making MAAS truly stateless with respect to machines.

Again, this isn’t every interaction between the controllers and machines, but it gives you a good idea of how MAAS works. You’ll notice that the discussion above introduces a lot of new concepts, which you can study in more detail in the documentation.