Learning About Amazon Linux 2023

References

• Amazon Linux 2023 FAQs
• Amazon Linux 2023 Features
• What is Amazon Linux 2023?
• YUM Cheatsheet
• DNF Command Reference
• Changes Between DNF and YUM
• cloud-init documentation
• systemd-networkd package

Related

• Fedora Linux
• • Fedora Linux is a Linux distribution developed by the Fedora project. It was originally developed in 2003 as a continuation of the Red Hat Linux project. It is now the upstream source for CentOS Stream and Red Hat Enterprise Linux.
  • Since the release of Fedora 21 in 2015, three editions have been made available: personal computer, server, and cloud computing. This was expanded to five editions for containerization and Internet of Things as of the release of Fedora 37 in 2022.
  • The distribution used by Linus Torvalds
  • Amazon Linux 2023 is downstream of Fedora

• RPM Package Manager
• • RPM Package Manager (RPM) (originally Red Hat Package Manager) is a free and open source package management system. The name RPM refers to the .rpm file format and the package program itself. RPM was intended primarily for Linux distributions; the file format is the package format of the Linux Standard Base.

• YUM
• • The Yellowdog Update Modified (YUM) is a free and open-source command-line package-management utility for computers running the Linux operating system using the RPM Package Manager. YUM allows for automatic updates and package and dependency management on RPM-based distributions. Like the Advanced Package Tool (APT) from Debian, YUM works with software repositories (collections of packages), which can be accesses locally or over a network connection. YUM can perform operations such as:
  • • installing packages
    • deleting pacages
    • updating existing installed packages
    • listing available packages
    • listing installed packages

• dnf
• • DNF or Dandified YUM is the next-generation version of the Yellowdog Updater Modified, a package manager for .rpm-based Linux distributions.
  • Perceived deficiencies of yum (which DNF is intended to address) include poor performance, high memory usage, and the slowness of its iterative dependency resolution. DNF uses libsolv, an external dependency resolver.

• cloud-init
• • Cloud-init is the industry standard multi-distribution method for cross-platform cloud instance initialization. Ut is supported across all major public cloud providers, provisioning systems for private cloud infrastructure, and bare-metal installations.
  • On instance boot, cloud-init will identify the cloud it is running on, read any provided metadata from the cloud, and initialize the system accordingly. This may involve setting up the network and storage devices, configuring SSH access keys, and setting up many other aspects of a system. Later, cloud-init will parse and process any optional user or vendor data that was passed to the instance.

• Active Queue Management
• • In routers and switches, active queue management (AQM) is the policy of dropping packets inside a buffer associated with a network interface controller before that buffer becomes full, often with the goal of reducing network congestion or improving end to end latency. This task is performed by the network scheduler, which for this purpose uses various algorithms such as random early detection, explicit congestion notification, or controlled delay (CoDel). RFC 7567 recommends active queue management as a best practice.

YUM Commands

Information About Format Below:

• <text> - replace <text> with whatever the text inside the angle brackets describes
• [option1,option2,...] - certain commands have optional parameters that provide slightly different functionality

# YUM Queries
$ yum help # Display yum commands and  options
$ yum list [available,installed,all,kernel] # List package names from repositories
$ yum info <pakage name> # display information about a package
$ yum deplist <package name> # display the dependencies for a package
$ yum provides <path to file> # find pacjages that provide the queried file
$ yum search <package name> # Search package names and descriptions for them
$ yum updateinfo <package name> # Getinformation about available package updates
$ yum grouplist # List names of installed and available package groups
$ yum groupinfo <package group> # display description and contents of a package group
$ yum check-update # query repositories for available package updates
# Manage YUM Repositories
$ yum repolist # Display enabled software repositories
$ yum repoinfo <repository name> # Display information about enabled yum repositories
$ yum repo-pkgs <respository name> [list,install,remove] # Work with packages in a particular repository
$ yum makecache # Download yum repository data to cache
# Troubleshoot and maintain YUM
$ yum check # check the local RPM database for problems (runs for a long time)
$ yum history [list,info <transaction number>,undo <transaction number>,redo <transaction number>] # view and use yum transactions
$ yum clean [packages,all] # clear out cached package data
$ yum fssnapshot # list LVM snapshots (helps roll back after package updates)
$ yum fs [filters,documentation] # Act on filesystem (prevent doc or language file install on minimal systems)
# Install, remove, and upgrade packages with yum
$ yum install <package name>
$ yum update <package name>
$ yum update-to # Update one or more packages to a particular version
$ yum upgrade # update packages taking obsoletes into account
$ yum localinstall <file,url> # Instal a package from a local file, http, or ftp
$ yum downgrade <package name> # Downgrade a package to an earlier version
$ yum reinstall <package name> # Reinstall the current version of a package
$ yum swap <package name 1> <package name 2> # Remove package 1 and install package 2
$ yum erase <package name> # Erase a package (and possibly dependencies) from your system
$ yum remove <package name> # same as remove
$ yum autoremove <packagene name> # Same as erase, plus removes additional unneeded packages
$ yum groupinstall # Install all packages in the selected group

The Default Package Management Tool in Amazon Linux 2023 is DNF, not YUM

• DNF is a successor to YUM, the package management tool in AL2.
• DNF is similar to YUM in its usage. Many DNF commands and command options are the same as YUM commands. In a Command Line Interface (cli) command, in most cases, dnf replaces yum
• In Amazon Linux 2023, the yum command is still available, but as a pointer to the dnf command. So, when the yum command is used in the shell or in a script, all commands and options are the same as the DNF CLI.

Notes

Amazon Linux 2023 (AL2023) is a general-purpose rpm-based Linux distribution and a successor to Amazon Linux 2. AL2023 simplifies planning for operating system upgrades. Starting with AL2023, a new major version of Amazon Linux is released every 2 years, includes minor quarterly releases, and comes with five years of long tern support. Amazon Linux 2023 integrates with AWS services and is designed to be deployed at scale in the cloud.

NOTE: AL2023 is an abbreviation for Amazon Linux 2023

• AL2023 maintains its own release and support lifecycles independent of Fedora. AL2023 provides updated versions of open source software, a larger variety of packages, and frequent releases.

Customized cloud-init

• The cloud-init package is an open-source application that bootstraps Linux images in a cloud computing environment. AL2023 contains a customized version of cloud-init. With cloud-init, you can specify what occurs to your instance at boot time. When you launch an instance, you can use the user-data fields to pass actions to cloud-init. This means that you can use common Amazon Machine Images (AMI) for many use cases and configure them dynamically when you start an instance. AL2023 also uses cloud-init to configure the ec2-user account.
• AL2023 use the cloud-init actions in /etc/cloud/cloud.cfg.d and /etc/cloud/cloud.cfg. You can create you own cloud-init action files in the /etc/cloud/cloud.cfg.d directory. Cloud-init reads all the files in this directory in lexicographical order. Later files overwrite values in earlier files. When cloud-init launches an instance, the cloud-init package does the following configuration tasks:
• • Sets the default locale
  • Sets the hostname
  • Parses and handles user-data
  • Generates host private SSH keys
  • Adda a user's public SSH keys to .ssh/authorized_keys for easy login and administration
  • Prepares the repositories for package management
  • Handles package actions that are defined in user-data
  • Runs user scripts that are in user-data
  • Mounts instance store volumes, if applicable
  • • By default, if the ephemeral0 instance store volume is present and contains a valid file system, the instance store volume is mounted at /media/ephemeral0. Otherwise, it's not mounted.
    • By default, for the m1.small and c1.medium instance types, all swap volumes that are associated with the instance are mounted.
    • You can override the default instance store volume mount with the following cloud-init directive:

#cloud-config
mounts:
- [ ephemeral0 ]

• • When an instance launches, the instance store volumes that support TRIM aren't formatted. Before you can mount instance store volumes, you must partition and format instance store volumes
  • When you launch your instances, you can use the disk_setup module to partition and format your instance store volumes.
• See the User-Data Formats in the cloud-init documentation for more information about user-data formats

Networking Service

• The open source systemd-networkd is widely available in modern Linux distributions. The project uses a declarative configuration language that's similar to the rest of the systemd framework. It's primary configuration file types are .network and .link files.
• The amazon-ec2-network-utils package generates interface-specific configurations in the /run/systemd/network directory. These configurations enable both IPv4 and IPv6 networking on interfaces when they're attached to an instance. These configurations also install policy routing rules that help ensure that locally sourced traffic is routed to the network through the corresponding instance's network interface. These rules ensure that the right traffic is routed through the Elastic Network Interface (ENI) from the associated addresses or prefixes.
• You can customize this networking behavior by placing a custom configuration file in the /etc/systemd/network directory to override the default configuration settings contained in /run/systemd/network.
• The systemd.network documentation describes how the systemd-network service determines the configuration that applies to a specific interface. It also generates alternative names, known ad altnames, for the ENI-backed interfaces to reflect the properties of various AWS resources. These ENI-backed interface properties are the ENI ID and the DeviceIndex field of the ENI attachment. You can refer to these interfaces using their properties when using various tools, such as the ip command.
• AL2023 instance interface names are generated using the systemd slot naming scheme. For more information, see systemd.net naming scheme.
• Additionally, AL2023 uses the fq_codel active queue management network transmission scheduling algorithm by default. For more information, see CoDel overview.

Core Toolchain Packages

• A subset of packages in Amazon Linux is designated as core toolchain packages. As a major part of AL2023, core packages receive five years of support.
• These three core packages provide a system toolchain that is used to build most software in the Amazon Linux distribution:

systemd-networkd

• systemd-networkd is a system daemon that manages network configurations. It detects and configures network devices as they appear; it can also create virtual network devices. This service can be especially useful to set up complex network configurations for a container managed by systemd-nspawn or for virtual machines. It also works fine on simple connections.
• The systemd package is part of the default Arch installation and contains all needed files to operate a wired network. Wireless adapters can be set up by services.
• To use systemd-networkd, start/enable systemd-networkd.service

Note: Ensure that no other service that wants to configure the network is running; in fact, multiple networking service will conflict. Find a list of the currently running services with systemctl --type=service and then stop them.

• It is optional to also configure systemd-resolved which is a network name resolution service to local applications.
• The global configuration file in /etc/systemd/networkd.conf may be used to override some defaults only. The main configuration is performed per network device. Configuration files are located in /usr/lib/systemd/network/, the volatile runtime network directory /run/systemd/network/ and the local administration network directory /etc/systemd/network/. Files in /etc/systemd/network/ have the highest priority.
• There are three types of configuration files. They all use a format similar to systemd unit files.
• • .network files - They will apply a network configuration for a matching device.
  • .netdev files - They will create a virtual network device for a matching environment.
  • .link files - When a network device appears, udev will look for the first matching .link file.
• They all follow the same rules:
• • If all conditions in the [Match] section are matched, the profile will be activated
  • an empty [Match] section means the profile will apply in any case (can be compared to the * wildcard)
  • all configuration files are collectively sorted and processes in lexical order, regardless of the directory in which they live
  • Files with identical names replace each other
• Network files are files aimed at setting network configuration variables, especially for servers and containers.
• • See Section 2.1 in this documentation or this systemd.network documentation to see the sections and keys for network files.

CoDel (pronounced coddle)

• CoDel is a novel "no knobs", "just works", "handles variable bandwidth and RTT", and simple Active Queue Management algorithm.
• • It is parameter-less - no knobs are required for operators, users or implementers to adjust
  • It treats good queue and bad queue differently - that is, it keeps the delays low while permitting bursts of traffic
  • It controls delay, while insensitive to round-trip delays, link rates, and traffic loads
  • It adapts to dynamically changing link rates with no negative impact on utilization
• CoDel (the name comes from "controlled delay") was a fundamental advance in the state of the art of network of Active Queue Management in 2021.
• Immediately after codel, came "fq_codel" (Fair/Flow Queueing + Codel) invented by Eric Dumazet. The combination made it possible to reduce bottleneck delays by several orders of magnitude and provide accurate RTT estimates to elephant TCP flows, while allowing shorter (sparser) flows like DNS, ARP, SYN, routing etc packets priority access.
• All Linux Systems that use systemd, now default to fq_codel. That includes but is not limited to debian, Ubuntu, redhat, fedora, and arch.