Saturday, November 25, 2023

IMDSv2

v2 of the AWS "Instance Metadata Service" has been around for a while.

  • It is optional for now, but they will make it mandatory some time during 2024 [1].
  • The IMDS has to do with how EC2 instances get their metadata and IAM permissions.
  • If you're using a recent version of cloud-init, AWS CLI and SDKs, these should support v2.
  • By default, most instances that have been around for a while still use v1.

To list instances that are still using v1 [2]:

aws ec2 describe-instances \
  --filters "Name=metadata-options.http-tokens,Values=optional" \
  --query "Reservations[*].Instances[*].[InstanceId]" \
  --output text

To enable v2 on a per-instance basis:

aws ec2 modify-instance-metadata-options \
  --instance-id "${EC2_ID}" \
  --http-endpoint enabled \
  --http-tokens required

To change an AMI so that instances launched from it have v2 enabled:

aws ec2 modify-image-attribute \
  --image-id "${AMI_ID}" \
  --imds-support v2.0

You will have to change how you retrieve instance metadata URIs [3].

v1:

curl -s http://169.254.169.254/latest/meta-data/instance-id

v2:

TOKEN="$(curl -X PUT "http://169.254.169.254/latest/api/token" -H "X-aws-ec2-metadata-token-ttl-seconds: 21600")"

curl -s -H "X-aws-ec2-metadata-token: $TOKEN"  \

Thursday, November 9, 2023

Debugging Linux input events

I recently had some issues with a laptop running Linux receiving spurious keyboard events.
The following two commands are useful in debugging this:

sudo tail -f /dev/input/event*

And:

xinput test-xi2 --root 5

You can replace the "5" with any other value to view input from other devices.


Saturday, November 4, 2023

Is your AWS instance Xen or Nitro?

This command will let you know whether the AWS instance you are logged into is Xen or Nitro-based:

curl http://169.254.169.254/latest/meta-data/system/

AWS has a page about Nitro, but not one about Xen, as that is considered a legacy virtualizer:
https://aws.amazon.com/ec2/nitro/

Check for Percona 8.0 upgrades

One thing I wasn't aware of is that in order to fully check for Percona upgrades, I had to manually run the command:

sudo percona-release setup ps80

https://docs.percona.com/percona-server/8.0/installation.html

Shrink AWS EBS root drive with UEFI boot mode

Recently I wanted to create a Ubuntu 22.04LTS image for the aarch64 (ARM) infrastructure with 3GB of space (instead of the default 8GB). I had done a similar procedure years ago, on a x64 Ubuntu installation, but this time the operation presented complications. Specifically, the default boot mode for Graviton instance types is UEFI: https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/ami-boot.html so in addition to the Linux partition, there is also an EFI partition that has to be handled.

The following should help you in case you want to achieve something similar.

Prerequisites:

  1. A working "source" instance , which contains the "source" volume (the one you want to shrink).
  2. A new empty volume, which we will call the "target" volume.
  3. A throwaway "staging" instance, to which we will attach both the "source" and the "target" volumes.
    (Even though the volume I was shrinking contained an ARM installation, I used an x64 image for the staging instance.)
The process doesn't actually shrink the volume. What happens instead is that the data we will be mirrored to the smaller drive.
The reason why a throawaway instance is preferred is that we don't want to accidentally risk affecting production machines and data.

Initial steps:
  • Before dismounting the source volume, clean the instance of any unnecessary files. You want the target volume to be able to fit all the data in the source volume (I removed Snap entirely and obsolete kernels to get below 3GB).
  • Detach the source volume from the source instance.
  • Attach the source volume to the staging instance (as /dev/sdf).
  • Attach the target volume to the staging instance (as /dev/sdg).
  • Boot the staging instance, SSH into it, and get a root bash prompt with sudo su.
Once inside, df -h should show the attached volumes.
On my particular staging instance, they showed up as /dev/xvdf and /dev/xvdg respectively.
Other guides have them as /dev/nvme1n1 and /dev/nvme2n1. This has to do with whether the staging instance is a Xen (xvd*) or Nitro (nvme*) one. Adjust as needed, but I personally prefer the Xen nomenclature for this operation. You're not going to mistake "f" and "g".

Step 1
Inspect the partitioning of the source volume:
(Highlights mine, those are values we're are interested in).

fdisk -l /dev/xvdf
Disk /dev/xvdf: 8 GiB, 8589934592 bytes, 16777216 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disklabel type: gpt
Disk identifier: 51ED0604-9F4E-4D14-B8E7-4FC378A5E100

Device       Start      End  Sectors  Size Type
/dev/xvdf1  206848 16777182 16570335  7.9G Linux filesystem
/dev/xvdf15   2048   204800   202753   99M EFI System

Partition table entries are not in disk order.

Notice that there are two partitions:
  • Partition 15 is at the start of the volume, and holds the UEFI files.
  • Partition 1 is at the end of the volume, and holds the Linux installation.
  • Make note of the total # of sectors, as well as the "End" and "Sectors" value for the Linux filesystem partition.

Step 2
Inspect the target volume:

fdisk -l /dev/xvdg
Disk /dev/xvdg: 3 GiB, 3221225472 bytes, 6291456 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disklabel type: dos
Disk identifier: 0x00000000

This volume doesn't have any partitions yet, so just make note of the total # of sectors.

Step 3
Dump the partition table of the source volume:

sudo sfdisk -d /dev/xvdf > partitions.txt
label: gpt
label-id: 51ED0604-9F4E-4D14-B8E7-4FC378A5E100
device: /dev/xvdf
unit: sectors
first-lba: 34
last-lba: 16777182
sector-size: 512

/dev/xvdf1 : start=      206848, size=    16570335, type=0FC63DAF-8483-4772-8E79-3D69D8477DE4, uuid=69E1997E-A903-4643-A4FA-491F49342E1C
/dev/xvdf15 : start=        2048, size=      202753, type=C12A7328-F81F-11D2-BA4B-00A0C93EC93B, uuid=CC54DFDE-E8B4-43AA-B808-0CBE0BDB7C0D

Notice that the value for "last-lba" and "size" are exactly the values from Step 1 for "End" and "Sectors".

Step 4
Calculate the new partition configuration.
We will leave partition 15 exactly as it is, what we're interested in is making sure partition 1 gets a valid partition configuration, so the partition is created perfectly the first time around.

If we look at Step 1, we notice that the "End" and "Sectors" values for /dev/xvdf1 are correlated to the total number of sectors of the disk.

Specifically:
"End" = disk sectors (16777216) - 34 = 16777182
"Sectors" = disk sectors (16777216) - 206881 = 16570335

To get the values for /dev/xvdg we simply substitute the total number of disk sectors:
"End" = 6291456 - 34 = 6291422
"Sectors" = 6291456 - 206881 = 6084575

On your volume the deltas may be different, adjust as needed.

Step 5
Edit the partition table dump file (partitions.txt) by filling in the values calculated at the previous step. The values that need to be changed are:

  • "last-lba" = the calculated "End" value.
  • "size" = the calculated "Sectors" value.
I also substituted "xvdf" for "xvdg" to avoid confusion when looking at the file, but I think only the numerical values are relevant. Other guides also suggest changing the uuid of the partitions, but here it's critical that we keep them.

The edited "partitions.txt":

label: gpt
label-id: 51ED0604-9F4E-4D14-B8E7-4FC378A5E100
device: /dev/xvdg
unit: sectors
first-lba: 34
last-lba: 6291422
sector-size: 512

/dev/xvdg1 : start= 206848, size= 6084575, type=0FC63DAF-8483-4772-8E79-3D69D8477DE4, uuid=69E1997E-A903-4643-A4FA-491F49342E1C
/dev/xvdg15 : start= 2048, size= 202753, type=C12A7328-F81F-11D2-BA4B-00A0C93EC93B, uuid=CC54DFDE-E8B4-43AA-B808-0CBE0BDB7C0D

Step 6
Write the new partition table to the target disk:

sudo sfdisk /dev/xvdg < partitions.txt

Step 7
Make a raw low-level copy of the EFI partition, which will remain unchanged:

sudo dd if=/dev/xvdf15 of=/dev/xvdg15 bs=512

Step 8
Mirror the Linux partition data by mounting the partition and copying the files with rsync:

mkdir /tmp/xvdf1
mkdir /tmp/xvdg1

mount /dev/xvdf1 /tmp/xvdf1
mount /dev/xvdg1 /tmp/xvdg1

rsync -av /tmp/xvdf1/ /tmp/xvdg1/

umount /dev/xvdf1
umount /dev/xvdg1

Step 9
Ensure source and target partitions have the same metadata.
First, query each of them using blkid:

blkid /dev/xvdf1
/dev/xvdf1: LABEL="cloudimg-rootfs" UUID="466bbba3-507d-44a0-989e-e25286198eba" BLOCK_SIZE="4096" TYPE="ext4" PARTUUID="69e1997e-a903-4643-a4fa-491f49342e1c"

blkid /dev/xvdg1
/dev/xvdg1: UUID="69e1997e-a903-4643-a4fa-491f49342e1c" BLOCK_SIZE="4096" TYPE="ext4" PARTUUID="69e1997e-a903-4643-a4fa-491f49342e1c" 

Notice that the source partition has a LABEL, and a different UUID.
Without mirroring these as well, the machine won't boot (the label is referenced by /etc/fstab, and the UUID is referenced by the UEFI configuration).

The following commands set the right values:

e2label /dev/xvdg1 cloudimg-rootfs
tune2fs /dev/xvdg1 -U 466bbba3-507d-44a0-989e-e25286198eba

One or both may ask you to run some integrity utility.
Do so, it should run without problems.

Step 10
Wrapping up:
  • Stop the machine
  • On the EC2 "Volumes" page detach the volumes from the staging instance.
  • Attach the target volume to the source machine as /dev/sda1
  • Boot.

Friday, November 3, 2023

screen - tmux dictionary

The following lists equivalent commands I most commonly use for the screen and tmux programs.
This is relevant because Red Hat has stopped making screen easily available, first marking it as deprecated in v7.6, then not even including it after v8.0: https://access.redhat.com/solutions/4136481

1. List existing sessions:

  • screen -ls
  • tmux ls # tmux list-sessions
2. Create a new named session:
  • screen -S name
  • tmux new -s name # or: tmux new-session -s name
3. Attach a named session:
  • screen -dr name
  • tmux a -t name # or: tmux attach -t name
4. Detach an attached session:
  • Ctrl+a d # screen
  • Ctrl+b d # tmux
5. Enter copy mode:
  • Ctrl+a Esc # screen
  • Ctrl+b [   # tmux
6. Exit copy mode:
  • Esc # screen
  • q   # tmux
7. Terminate named session:
  • tmux kill-session -t name
  • screen -XS name
8. Start a command in a session:
  • screen -dmS session_name command param1 param2
  • tmux new -d -s name command param1 param2

Wednesday, November 1, 2023

Bing: disable scrolling from Bing AI chat to search results

For some reason, Microsoft introduced a UX feature whereby the page would switch:

  • from Bing AI to the search results page when scrolling down, and
  • from the search results page to Bing AI when scrolling up.

This happened even though the scrollbar gave no indication there was additional content to scroll to.
The following userscript, simplified from a version I found on the Microsoft Community forum, disables this behavior:

// ==UserScript==
// @name Bing scroll
// @match https://www.bing.com/*
// ==/UserScript==

window.addEventListener("wheel", e => {
  if (e.target.className.includes("cib-serp-main")) {
    e.stopPropagation();
  }
});

Original:
https://answers.microsoft.com/en-us/bing/forum/all/how-to-disable-scrolling-into-ai-chat/2c208d88-918f-4eed-bd8f-f04e7dcf5af1?page=5