Running Chrome OS in LAVA

Running Chrome OS images and dedicated Chrome OS tests come with very specific requirements. These tests typically can’t be run with any other user-space than Chrome OS and have to be deployed via the Tast framework.

Flashing a Chrome OS image on a Chromebook

The first step is to flash a Chrome OS image onto a Chromebook, as they can’t be booted over NFS. This can be done within a Debian Buster NFS root with no special tools installed. It can be entirely automated in a LAVA job. Once the device has booted, the Chrome OS image needs to be downloaded (public location yet to be determined). Then here’s a sample script to flash the image from within an NFS root job, assuming the image is called octopus-chromiumos-test-image.bin:

cd /root
mkdir chromeos
dev=$(losetup -P -f --show octopus-chromiumos-test-image.bin)
mount "$dev"p3 chromeos -o ro
cd chromeos
mount udev -t devtmpfs dev
mount proc -t proc proc
mount sysfs -t sysfs sys
mount -t tmpfs tmpfs tmp
mount --bind /root/ mnt/empty
mount
ls -l mnt/empty/octopus-chromiumos-test-image.bin
ls -l dev/mmc*
echo "Starting to flash..."
chroot . \
  /usr/sbin/chromeos-install \
  --dst /dev/mmcblk0 \
  --payload_image /mnt/empty/octopus-chromiumos-test-image.bin \
  --yes

It relies on the chromeos-install script which is provided inside the image itself. This is done by accessing the main partition (read-only) in the image via losetup and chroot. The image file is made available “within itself” by bind-mounting it in a known empty directory /mnt/empty.

Rootfs for flashing Chrome OS using Debos

Rootfs image recipe can be obtained by following below steps

$ git clone https://gitlab.collabora.com/lava/collabora-lava-setup
$ sudo docker run -it -v $(pwd):/chromeos --device /dev/kvm --privileged -w /chromeos kernelci/debos bash
# debos tools/chromeos/chromeos-rootfs.yml

and resulting image can be found at /chromeos/debian-buster-arm64.tgz.

Flashing a Chrome OS image using LAVA:

Once rootfs is ready, then we need to add Chrome OS image file added by hand since its not available via HTTP server yet. Finally we can run LAVA job to flash Chrome OS image on octopus device. The flashing script chromeos-flash.sh looks for the image in /root directory of the rootfs. By default, the chromeos-flash.sh script, uses chromiumos_test_image.bin but you can also pass specific image as a CLI argument e.g. chromeos-flash.sh my_image_file.bin.

LAVA job definition for flashing ChromeOS can be found below and its taken from here

device_type: hp-x360-12b-ca0500na-n4000-octopus

context:
  extra_kernel_args: console_msg_format=syslog cros_secure

job_name: Chrome OS boot 5.10 - octopus
timeouts:
  job:
    minutes: 60
  action:
   minutes: 60
  actions:
    power-off:
      seconds: 30
priority: high
visibility: public


actions:
- deploy:
    timeout:
      minutes: 15
    to: tftp
    kernel:
      url: https://storage.kernelci.org/stable/linux-5.10.y/v5.10.50/x86_64/x86_64_defconfig+x86-chromebook/gcc-8/kernel/bzImage
    modules:
      url: https://storage.kernelci.org/stable/linux-5.10.y/v5.10.50/x86_64/x86_64_defconfig+x86-chromebook/gcc-8/modules.tar.xz
      compression: xz
    nfsrootfs:
      url: file:///tmp/debian-buster-amd64.tar
    ramdisk:
      url: http://storage.kernelci.org/images/rootfs/debian/buster/20210503.0/amd64/initrd.cpio.gz
      compression: gz
    os: oe


- boot:
    timeout:
      minutes: 15
    method: depthcharge
    commands: nfs
    prompts:
      - '/ #'


- test:
    timeout:
      minutes: 60
    definitions:
    - repository:
        metadata:
          format: Lava-Test Test Definition 1.0
          name: hack
          description: "hack"
          os:
            - debian
          scope:
            - functional
          environment:
            - lava-test-shell
        run:
          steps:
            - lsblk
            - ls -l /root
            - /bin/bash /usr/bin/chromeos-flash.sh
      lava-signal: kmsg
      from: inline
      name: hack
      path: inline/hack.yaml

Booting with Chrome OS

Some particular kernel configurations need to be enabled in order for all the Chrome OS services to fully boot.

To be continued

Running tests with Tast

Once the device has booted Chrome OS, it’s possible to start using Tast over SSH from a separate host (e.g. a LAVA dispatcher). Tast can be run within Docker on the host, and this is the approach take with LAVA jobs too.

To be continued

Running Chrome OS on QEMU

It is possible to run Chrome OS using test image on QEMU. A few steps need to be completed to achieve this:

Get Chrome OS SDK up and running
Get Chrome OS test image
Convert test image to QEMU raw image
Convert raw image to qcow2 (optionally)
Run ChromeOS with QEMU

Note This operation requires about 30GB of disk space. Make sure you have enough storage free before you start.

Get Chrome OS test image

Download octopus test image from https://share.collabora.com/index.php/f/1792000

This URL is not publicly available as the Chrome OS images are not shared publicly. This is supposed to be changed when either we get a permission to share the images or Google puts them in a public location.

Get Chrome OS SDK up and running

Chrome OS SDK is a Gentoo based chroot environment, which contains many scripts that facilitate development and building images. The following instructions describe how to setup Chrome OS SDK on a Debian Buster based Linux box.

Install prerequisites

sudo apt-get update
apt-get install --no-install-recommends -y \
apt-transport-https \
ca-certificates

Install required packages

sudo apt-get install --no-install-recommends -y \
curl \
git \
gnupg \
lvm2 \
procps \
python-pkg-resources \
python3.6 \
ssh \
sudo \
thin-provisioning-tools \
xz-utils \
ssh \
locales-all

Get depot_tools

git clone --depth=1 \
https://chromium.googlesource.com/chromium/tools/depot_tools.git

Add depot_tools to your PATH

export PATH=$PWD/depot_tools:$PATH

Note It may come in handy to add it to your .bashrc

Create a workspace directory

It’s convenient to keep your work in a separate directory.

mkdir chromiumos
cd chromiumos

Get source code repo manifest and install SDK

repo init \
  -u https://chromium.googlesource.com/chromiumos/manifest.git \
  -b release-R94-14150.B \
  --depth=1
repo sync -j$(nproc)

Create cache and chroot directories

mkdir chromiumos/cache
mkdir chromiumos/octopus-chroot

Create and enter chroot

cros_sdk \
    --enter \
    --nouse-image \
    --no-ns-pid \
    --debug \
    --chroot chromiumos/octopus-chroot \
    --cache-dir chromiumos/cache \

Convert test image to QEMU raw image

Copy Chrome OS image zip file to the location accessible from CrOS chroot
Enter Chrome OS SDK chroot (if not already in)

cros_sdk \
    --enter \
    --nouse-image \
    --no-ns-pid \
    --debug \
    --chroot chromiumos/octopus-chroot \
    --cache-dir chromiumos/cache \

After entering the chroot you’ll be put in the ~/chromiumos/src/scripts directory. All further instructions assume paths relative to this one.

Create images directory and unzip the test image files there

cp ChromeOS-R94-14150.19.0-octopus.zip $PWD/octopus-chroot/home/<your_username>/

mkdir images
cd images
unzip /home/<your_username>/ChromeOS-R94-14150.19.0-octopus.zip

use image_to_vm.sh script to convert test image to QEMU image

cd ~/chromiumos/src/scripts
./image_to_vm.sh --test_image --from images/

After a successful run /home/cros-build/images/chromiumos_qemu_image.bin file should be created.

Running QEMU image with kernelci/qemu docker container

It is possible to use chromiumos_qemu_image.bin with QEMU from kernelci/qemu docker image. It may be convenient in case you want to run it in LAVA.

Make sure you can access chromiumos_qemu_image.bin and id_rsa from outside of the chroot.
Make sure that id_rsa has right permissions

chown 600 /path/to/images/id_rsa

Run the container and enter bash shell

docker container run -it --device /dev/kvm -v /path/to/images:/images -p 2222:2222/tcp -p 5900:5900/tcp kernelci/qemu /bin/bash

Convert raw QEMU image to qcow2

qemu-img create -f qcow2 -o 'backing_file=/images/chromiumos_qemu_image.bin,backing_fmt=raw' /images/qcow2.img

Run QEMU VM

qemu-system-x86_64 -m 4096 -smp 4 -vga virtio -cpu SandyBridge,-invpcid,-tsc-deadline,check -usb -device usb-tablet -device 'virtio-net,netdev=eth0' -device 'virtio-scsi-pci,id=scsi' -device virtio-rng -device 'scsi-hd,drive=hd' -drive 'if=none,id=hd,file=/images/qcow2.img,cache=unsafe,format=qcow2' -netdev 'user,id=eth0,net=10.0.2.0/27,hostfwd=tcp:127.0.0.1:9222-:22,hostfwd=tcp:0.0.0.0:2222-:22' -display 'vnc=0.0.0.0:0' -enable-kvm

Access Chrome OS QEMU VM

It is possible to access QEMU VM running Chrome OS over SSH. It is also possible to access the VM display over VNC.

Access Chrome OS shell over SSH

ssh  -oStrictHostKeyChecking=no -oUserKnownHostsFile=/dev/null -p 2222 -i /path/to/unpacked/images/id_rsa root@localhost

Note While it’s preferred to use the key provided with the Chrome OS to log in, it’s still possible to authenticate with a password. The default password is test0000

Access VM display over VNC

It’s a known issue that no console appears over VNC and the last message is “Booting the kernel.”. It’s to be investigated in future.

vncviewer localhost:5900