Troubleshooting common MAAS issues
Tackle common MAAS issues
This guide helps you navigate through common MAAS challenges, organised by topic:
- Fix leaked MAAS admin API key
- Networking issues
- Machine life-cycle failures
- Custom image issues
- Session timeouts
- Other miscellaneous issues
Fix leaked MAAS admin API key
If MAAS hardware sync leaks your admin API key, you can:
- Rotate all admin tokens
- Re-deploy machines with hardware sync enabled
Or swap the token manually:
Manually swap the MAAS admin API token
Query the database to identify machines with hardware sync enabled:
select system_id
from maasserver_node
where enable_hw_sync = true;
Rotate API keys on any affected machines. To verify an API key belongs to an admin, perform this database query:
select u.username, u.email
from auth_user u
left join piston3_consumer c
on u.id = c.user_id
where key = 'your-leaked-api-key';
To remove the leaked API key, log in to the MAAS UI and delete it. Then reconfigure your MAAS CLI and hardware sync as needed.
Networking issues
Address networking issues like overlapping subnets and IP reconfiguration.
Overlapping subnets can break deployments
Ensure that your subnets don’t overlap to avoid deployment failures. Check and delete any outdated or redundant subnets through the Web UI.
Need to reconfigure server IP address
If you need to modify your MAAS server’s IP, simply re-run the setup:
sudo dpkg-reconfigure maas-region-controller
Network booting IBM Power servers
IBM Power servers with OPAL firmware utilise Petitboot for PXE interactions. For smooth deployment, configure a specific NIC as the network boot device via Petitboot.
Resolve DNS conflicts between LXD and MAAS
If both MAAS and LXD are managing DNS, disable LXD’s DNS and DHCP:
lxc network set $LXD_BRIDGE_NAME dns.mode=none
lxc network set $LXD_BRIDGE_NAME ipv4.dhcp=false
lxc network set $LXD_BRIDGE_NAME ipv6.dhcp=false
Machine life-cycle failures
Understand and fix machine life-cycle issues.
Nodes hang on “Commissioning”
Timing issues: Make sure the hardware clocks on your nodes and MAAS server are synchronised.
Network drivers: Use Linux-compatible network adaptors if commissioning hangs due to driver issues.
Feel free to contribute additional issues and solutions.
Custom image issues
Command ‘packer’ not found
When you try to run packer
or execute a make
command, you may encounter an error message indicating that packer
is not installed. The issue can be resolved by following these instructions.
Error with packer
:
stormrider@neuromancer:~$ packer
Command 'packer' not found...
Error with make
:
stormrider@neuromancer:~/mnt/Dropbox/src/git/packer-maas/ubuntu$ make
sudo: packer: command not found...
No rule to make target …OVMF_VARS.fd
Should you see an error like the one below, you’ve forgotten to install a needed dependency.
make: *** No rule to make target '/usr/share/OVMF/OVMF_VARS.fd'...
Failure to create QEMU driver
Encountering the following error means you’re missing a dependency. Once again, refer to this guide for resolution.
Failed creating Qemu driver: exec: "qemu-img": executable file not found in $PATH
Session timeouts
Timeout changes not taking effect
If you’ve modified the session timeout settings in the MAAS web interface but don’t see the changes, do the following:
- Make sure you’ve got administrative access to the MAAS web interface for changing session timeout settings.
- After altering the session timeout duration, don’t forget to save the new settings.
- Clear your browser’s cache and cookies. They might be holding on to old settings. Restart your browser and try again.
Users logged out before timeout expires
If users are getting logged out before the session timeout you’ve set, consider these checks:
- Double-check the unit of time you’ve set for the session timeout (weeks, days, hours, minutes). A mistake in units can cause unexpected timeouts.
- Inspect any server settings conflicting with MAAS that may cause premature session timeouts, like window manager logout settings in Ubuntu.
- If you’re using a load balancer or proxy, make sure it’s not causing additional timeouts conflicting with MAAS.
Can’t set an infinite session timeout
You can’t set an “infinite” session timeout in MAAS. The maximum allowed duration is 14 days. This limit strikes a balance between security and usability.
Users are suddenly logged out
MAAS will auto-logoff users when the session timeout duration is reached. If this happens more often than desired, consider increasing the timeout value to prevent frequent “idle-time” logouts.
Can’t set different timeouts for user groups
MAAS only supports a global session timeout setting. While you can’t customise this by user group, you could deploy separate MAAS instances with different configurations to achieve similar effects.
Can’t extend sessions beyond the timeout
The timeout duration resets every time there’s activity from the user. To extend a session, simply refresh the page before the timeout period ends. This will reset the session timer.
Other miscellaneous issues
In your MAAS journey, you might stumble upon issues that don’t easily fit into predefined categories. Here are some examples:
- Django sub-architecture error
- Forgotten MAAS admin password
- Missing MAAS web UI
- Backdoor image login
- Migrating snap installations
- Manual MAAS database export
Feel free to contribute additional issues and solutions you may encounter.
Django sub-architecture error
Sometimes, you may face the following Django error:
django.core.exceptions.ValidationError: ['Subarchitecture(<value>) must be generic when setting hwe_kernel.']
To solve this, try specifying a different commissioning kernel—perhaps upgrading from Xenial to Focal.
Forgotten MAAS admin password
If you forget your MAAS admin password but have sudo privileges, you can reset it like so:
sudo maas changepassword $PROFILE
Replace $PROFILE
with the username.
Missing MAAS Web UI
The default MAAS web UI is at http://<hostname>:5240/MAAS/
. If it’s unreachable:
- Verify Apache is running:
sudo /etc/init.d/apache2 status
. - Validate the hostname or try
http://127.0.0.1:5240/MAAS/
.
Backdoor image login
Ephemeral images boot nodes during MAAS activities. If you need emergency access, you can create a temporary backdoor in these images. This lets you log in to check logs and debug issues.
Extract the cloud image
Download the appropriate image and extract its files:
wget https://cloud-images.ubuntu.com/xenial/current/xenial-server-cloudimg-amd64-root.tar.gz
mkdir xenial
sudo tar -C xenial -xpSf xenial-server-cloudimg-amd64-root.tar.gz --numeric-owner --xattrs "--xattrs-include=*"
Generate password hash
Create a SHA-512 hashed password:
python3 -c 'import crypt; print(crypt.crypt("ubuntu", crypt.mksalt(crypt.METHOD_SHA512)))'
Modify the xenial/etc/shadow
file to insert this hash.
Rebuild squashfs image
Create a new SquashFS file with your changes:
sudo mksquashfs xenial/ xenial-customized.squashfs -xattrs -comp xz
Replace the existing MAAS image with this customised one.
Migrating snap installations
For snap-based MAAS in ‘all’ mode, you can migrate to a local PostgreSQL:
sudo /snap/maas/current/helpers/migrate-vd Snapatabase
Manual MAAS database export
To manually move your MAAS database, run:
export PGPASS=$(sudo awk -F':\\s+' '$1 == "database_pass" {print $2}' \
/var/snap/maas/current/regiond.conf)
sudo pg_dump -U maas -h /var/snap/maas/common/postgres/sockets \
-d maasdb -F t -f maasdb-dump.tar
Stop the MAAS snap (sudo snap stop maas
) and create a new PostgreSQL user and database for MAAS on the destination machine.
This should cover various miscellaneous issues you may encounter while using MAAS. Feel free to contribute with your own experiences.
Last updated a month ago.