bachelor/manual/manual.tex

\documentclass[12pt,a4paper]{article}

\usepackage[utf8]{inputenc}
\usepackage[english]{babel}
\usepackage[english]{isodate}
\usepackage[parfill]{parskip}
\usepackage{xurl}
\usepackage{graphicx}
\usepackage{float}

\begin{document}

\begin{titlepage}
    \begin{center}
        \vspace*{1cm}
            
        \Huge
        \textbf{MiRiV OS Extensions}
            
        \vspace{0.5cm}
        \LARGE
        User Guide
            
        \vspace{1.5cm}
            
        \textbf{Norbert Tremurici (11907086)}
            
        \vfill
            
        A companion to the bachelor thesis,\\
        ``Exploring extensions for a basic RISC-V pipeline implementation to support the execution of an operating system''
            
        \vspace{0.8cm}
            
        \Large
        Institute of Computer Engineering, TU Wien, Austria\\
        September 6, 2023
            
    \end{center}
\end{titlepage}

\newpage
\pagenumbering{Roman}
\tableofcontents
\newpage
%\listoffigures
%\newpage
%\listoftables
%\newpage
\pagenumbering{arabic}

\section{Introduction}

This document serves as an extension to the bachelor thesis, such that all relevant information that would become necessary to use the end result or further extend it remains available and documented.

All source code, for the implementation itself but also the thesis, resides in two separate repositories.
They are called \texttt{riscv\_bt}\footnote{\url{https://notabug.org/strlst/riscv_bt}} for the code and \texttt{bachelor}\footnote{\url{https://notabug.org/strlst/bachelor}} respectively.
Access is restricted but can be granted on request after obtaining permission.

\newpage
\section{The Documentation and Data Repository}

The repository \texttt{bachelor} contains the source definitions of this manual, a summary, a presentation and the thesis.
But it also contains miscellaneous directories with diagrams, notes and literature.
The most important directory for the purposes of this user guide is \texttt{linux}, which houses analysis data and scripts to generate or process them.
We will now restrict ourselves to this directory.

In order to systematically choose an appropriate operating system target and identify its requirements, a suite of scripts that process compiled binaries was created.
These binaries were the result of compilation processes specific to the operating systems that were chosen for detailed analysis.
These were \textit{embedded Linux}, \textit{FreeRTOS} and \textit{Zephyr RTOS} respectively.

For \textit{embedded Linux}, buildroot has been used (both as a community fork version that supports otherwise unimplemented features, but also an official version).
The produced version of Linux has been stripped as much as is possible, to target as minimal a set of requirements as is possible without further modification.
As a result of the buildroot compilation process, which is further described in the thesis, a file \texttt{vmlinux} is produced.
This file is an uncompressed, statically linked executable that contains the Linux kernel.
Additionally, within the root file system, a binary containing the user space busybox executable (\texttt{rootfs/bin/busybox}) is also contained.

As for \textit{FreeRTOS} and \textit{Zephyr RTOS}, they cannot be customized easily using a tool like buildroot but are rather defined by code, their source definitions reside in the other repository.
Out of these definitions, executable files are generated.

All resulting executable files are decomposed using \texttt{objdump}, such that assembly code is created.
For the purposes of the scripts those binaries are assumed to be have been copied to subdirectories \texttt{rv32-glibc-busybox-linux}, \texttt{rv32-uclibc-busybox-nommu-linux}, \texttt{rv32-freertos} and \texttt{rv32-zephyr}.

\newpage
\subsection{Scripts}

The following scripts exist and can be used:

\subsubsection{Script \texttt{filter\_process\_assembly}}

This script takes as arguments an \textit{input assembly source file}, \textit{output instruction file} and \textit{output csr file}, to decompose the input files into the two output files.

Example usage:

\texttt{mkdir -p dump}

\texttt{./filter\_process\_assembly main.elf.s dump/main.instr dump/main.csr}

\subsubsection{Script \texttt{filter\_csr\_reg.py}}

This script is used to extract csr registers, it is wrapped by \texttt{filter\_process\_assembly} due to complicated usage.

\subsubsection{Script \texttt{objdump}}

This script is a wrapper of the \texttt{objdump} binary, taking as an argument its \textit{parameters}, the \textit{executable file} and the \textit{target assembly file}.

Example Usage: \texttt{./objdump "-d" vmlinux dump/vmlinux.S}

\subsubsection{Scripts \texttt{dump\_[linux|freertos|zephyr]}}

These scripts use scripts \texttt{filter\_process\_assembly} and \texttt{objdump} to automatically process the executable files within the directory structure.
They disassemble the executable file in the current working directory and then apply the script to generate files of the analysis.

Example Usage: \texttt{./dump\_linux}

\subsubsection{Script \texttt{run}}

This script is used to test the generated Linux image in a virtual machine using \texttt{qemu}.

Example Usage: \texttt{./run}

\subsubsection{Script \texttt{filter\_merge\_instr\_files}}

This script is used to take the analysis files (\texttt{*.instr.*}) and generate total instructions counts.
It does so by wrapping the script \texttt{filter\_collect\_instr\_files.py}.
The directories that are used for analysis are static, although execution of the script can still easily be changed, because the script defines shell functions which can be easily reused.
As output, count files (\texttt{count-*-base}, \texttt{count-*-ext}) with total instruction counts are created.

Example Usage: \texttt{./filter\_merge\_instr\_files}

\subsubsection{Script \texttt{filter\_collect\_instr\_files.py}}

This script is used to transform analysis files (\texttt{*.instr.*}), it is wrapped by \texttt{filter\_process\_assembly} due to complicated usage.

\subsubsection{Script \texttt{histogram.py}}

This script is used to generate histogram plots out of the total instruction count files.
The files that are used are statically embedded, but can easily be changed by modifying the lists \texttt{names}, \texttt{filepaths1} and \texttt{filepaths2}.

Example Usage: \texttt{./histogram.py}

\newpage
\section{The Processor Implementation Repository}

The processor implementation repository is an extension of and closely resembles the MiRiV project template that has been used for the lab course \textit{Digital Design and Computer Architecture}.
This is no accident of course, as one of the stated goals of the thesis was to provide extensions that students can make use of within their own implementation.
Currently the repository has multiple \texttt{git} branches.
Due to previous collaboration with a colleague, development has mainly taken place in branch \texttt{osdev}.
After development has resulted in a working prototype and finally an implementation with (known) bugs fixed, the code was subsequently refactored so as to extract the additional code into separate VHDL files.
Branch \texttt{refactor} represents a first but non-functional attempt, while \texttt{refactor2} is a successful attempt at extracting all additions and has since been merged into branch \texttt{master}.

\newpage
\subsection{VHDL Files}

Directory \texttt{miriv/vhdl} contains all hardware definitions, beginning with code of the sample pipeline and cache implementation that was kindly provided.
Subdirectory \texttt{miriv/vhdl/osext} contains all extensions, either in VHDL packages, individual units or pipeline stage extensions.

The packages are:

\begin{itemize}
    \item{core package (\texttt{miriv/vhdl/osext/osext\_core\_pkg.vhd})}
    \item{operations package (\texttt{miriv/vhdl/osext/osext\_op\_pkg.vhd})}
\end{itemize}

The individual units are:

\begin{itemize}
    \item{a branch predictor unit (\texttt{miriv/vhdl/osext/bp.vhd})}
    \item{CLINT unit (\texttt{miriv/vhdl/osext/clint.vhd})}
    \item{CSR unit (\texttt{miriv/vhdl/osext/csr.vhd})}
    \item{multi-cycle integer division unit (\texttt{miriv/vhdl/osext/intdiv.vhd})}
    \item{multi-cycle muldiv units (preferrably \texttt{miriv/vhdl/osext/muldiv2.vhd}, alternate implementation in \texttt{miriv/vhdl/osext/muldiv.vhd})}
    \item{trap unit (\texttt{miriv/vhdl/osext/trap.vhd})}
\end{itemize}

The pipeline stage extensions are:

\begin{itemize}
    \item{fetch stage extension (\texttt{miriv/vhdl/osext/osext\_fetch.vhd})}
    \item{decode stage extension (\texttt{miriv/vhdl/osext/osext\_decode.vhd})}
    \item{execute stage extension (\texttt{miriv/vhdl/osext/osext\_exec.vhd})}
    \item{memory stage extension (\texttt{miriv/vhdl/osext/osext\_mem.vhd})}
\end{itemize}

\newpage
\subsection{Software Files}

Directory \texttt{miriv/software} contains all code to be executed, either in simulations or on the FPGA targets after download.
As before, subdirectories \texttt{miriv/software/asm} and \texttt{miriv/software/c} contain assembly and c code respectively, but new directories \texttt{miriv/software/freertos} and \texttt{miriv/software/zephyr} are also available.

\subsubsection{FreeRTOS}

FreeRTOS has its kernel space code placed in \texttt{miriv/software/freertos/source} and user space code in \texttt{miriv/software/freertos/demo/rv32izicsr-miriv-gcc}.
Kernel space code has been left untouched and as such could be updated to newer versionso of FreeRTOS by replacing kernel space source files.
User space code has a copy of the utility library used for the C programs, used for example to write to UART, but also to access the hardware counters.
The following files are relevant:

\begin{itemize}
    \item{\texttt{miriv/software/freertos/demo/rv32izicsr-miriv-gcc/Makefile}: used to build the project}
    \item{\texttt{miriv/software/freertos/demo/rv32izicsr-miriv-gcc/crt0.S}: start-up code processor boots into}
    \item{\texttt{miriv/software/freertos/demo/rv32izicsr-miriv-gcc/util.h}: definitions of the utility library}
    \item{\texttt{miriv/software/freertos/demo/rv32izicsr-miriv-gcc/util.c}: implementation of the utility library}
    \item{\texttt{miriv/software/freertos/demo/rv32izicsr-miriv-gcc/main.c}: user space main implementation}
    \item{\texttt{miriv/software/freertos/demo/rv32izicsr-miriv-gcc/FreeRTOSConfig.h}: configuration file used to influence kernel and compilation behavior}
\end{itemize}

\subsubsection{Zephyr RTOS}

Zephyr RTOS re-uses definitions for other boards and has a very minimal configuration.
Whereas FreeRTOS has been tested to run in simulations and on the FPGA targets, the Zephyr RTOS project has only been developed to the point of compiling into an executable that can be analyzed for its requirements.
To build it, an environment in which its build tool \texttt{west}\footnote{\url{https://docs.zephyrproject.org/latest/develop/west/index.html}} can be used must be set up.
All relevant files can be found in \texttt{miriv/software/zephyr/threads}.

\newpage
\section{Writing FreeRTOS Programs}

FreeRTOS is an extensively documented platform, ported to many architectures and offers many features required of a real-time operating system.
Information about it can be found on the main page\footnote{\url{https://www.freertos.org/RTOS.html}}, but there are additional resources on how to get started\footnote{\url{https://www.freertos.org/FreeRTOS-quick-start-guide.html}}, developer documentation\footnote{\url{https://www.freertos.org/features.html}}, secondary documentation\footnote{\url{https://www.freertos.org/kernel/secondarydocs.html}}, information about supported devices\footnote{\url{https://www.freertos.org/RTOS_ports.html}} and an API reference\footnote{\url{https://www.freertos.org/a00106.html}}.

To get going and implement or extend the FreeRTOS demo, the most useful out of these resources are the getting started guide, developer documentation, secondary documentation and API reference.
Currently a simple two-task demo application is implemented.
The main entry point and implementation is contained in:

\texttt{miriv/software/freertos/demo/rv32izicsr-miriv-gcc/main.c}

Both tasks are defined within a function that are registered during task creation and the tasks are created statically.
Currently, functions to be executed as tasks are the tasks which internally count and print to UART (\texttt{task1} and \texttt{task2}), or the two C programs that run to completion and print intermediary results as tasks (\texttt{task\_primesieve} and \texttt{task\_matmul}).

Also important is the configuration of the FreeRTOS kernel itself.
The configuration file is contained in:

\texttt{miriv/software/freertos/demo/rv32izicsr-miriv-gcc/FreeRTOSConfig.h}

To see what the options mean and how they influence kernel or compilation behavior, consult the page in the developer documentation\footnote{\url{https://www.freertos.org/a00110.html}}.

Finally, the start-up code and utility library can be changed or extended, for example to interface with new IO devices.
These parts of the software represent the hardware abstraction layer for the MiRiV pipeline and bootloader.
They are contained in:

\texttt{miriv/software/freertos/demo/rv32izicsr-miriv-gcc/crt0.S}

\texttt{miriv/software/freertos/demo/rv32izicsr-miriv-gcc/util.[hc]}

\newpage
\section{Using FreeRTOS on Hardware}

To bring it all together, this section explains how to get the demo working on both of the available FPGA targets.
In order to set up, both the software and hardware need to be compiled/synthesized.

\newpage
\subsection{Compiling FreeRTOS}

To compile FreeRTOS, navigate to its demo directory and use the \texttt{Makefile}:

\texttt{cd miriv/software/freertos/demo/rv32izicsr-miriv-gcc}
\texttt{make clean all}

The \texttt{Makefile} will compile FreeRTOS and produce files \texttt{main.elf}, \texttt{main.imem.mif}, \texttt{main.dmem.mif} and \texttt{main.elf.s}.

The former file \texttt{main.elf} is an executable file, the \texttt{main.(imem|dmem).mif} are familiar files containing hexadecimal data used for programming the processor using JTAG and the \texttt{main.elf.s} file is a disassembled assembly file that is generated for analysis and debugging purposes (useful for exception handling or simulation, for example).

Additionally, two lines will indicate the resource usage of the \texttt{main.(imem|dmem).mif} files, which has to be watched not to exceed the resource limitations (16 KiB respectively) of the hardware to avoid problems.

\begin{figure}[h]
    \centering
    \includegraphics[width=\textwidth]{graphics/freertos-compilation-output.png}
    \caption{Sample screenshot of FreeRTOS compilation.}
\end{figure}

Additional \texttt{Makefile} targets \texttt{\%.instr} and \texttt{\%.csr} exist for analysis purposes.
Flags can also be set and paths changed to configure the compilation process, such as what RISC-V extensions to use where to source the RISC-V toolchain from.

\subsection{Synthesizing MiRiV}

To synthesize MiRiV, the bootloader has to be packed and the core synthesized.
Both the bootloader and core are managed using a \texttt{Makefile}.
When targeting the Terasic DE0-Nano board instead of the Terasic DE2-115 board, the flag \texttt{NANO\_BOARD} has to be set to \texttt{TRUE} inside the \texttt{Makefile}.
The bootloader inside the \texttt{miriv/bootloader} is packed as follows:

\texttt{cd miriv/bootloader}

\texttt{make clean pack}

\texttt{cd ../..}

\begin{figure}[h]
    \centering
    \includegraphics[width=\textwidth]{graphics/miriv-bootloader-packing.png}
    \caption{Sample screenshot of successful bootloader packing.}
\end{figure}

After packing, navigate to the \texttt{miriv/quartus} folder to synthesize the core:

\texttt{cd miriv/quartus}

\texttt{make clean all}

\begin{figure}[h]
    \centering
    \includegraphics[width=\textwidth]{graphics/miriv-core-synthesis.png}
    \caption{Sample screenshot of successful core synthesis.}
\end{figure}

\newpage
\subsection{Running FreeRTOS}

To view UART output of the demo application, open up a new terminal, navigate to the \texttt{miriv/quartus} folder and use the \texttt{minicom} target (after plugging in the appropriate USB connector):

\texttt{make minicom}

Finally, to run FreeRTOS, use the \texttt{download} target to download the \texttt{miriv.sof} file to the FPGA and then use the \texttt{run\_freertos} to write to instruction and data memory:

\texttt{make download}

\texttt{make run\_freertos}

\begin{figure}[H]
    \centering
    \includegraphics[width=\textwidth]{graphics/miriv-core-download.png}
    \caption{Sample screenshot of successful core download.}
\end{figure}

\begin{figure}[H]
    \centering
    \includegraphics[width=\textwidth]{graphics/miriv-core-run-freertos.png}
    \caption{Sample screenshot of successful core FreeRTOS run.}
\end{figure}

\newpage
\section{Using FreeRTOS in Simulations}

FreeRTOS can also be used as a target in simulations.
A helper script can be found at \texttt{miriv/sim/execute\_test.sh}, it has multiple options to run assembly targets, C targets or FreeRTOS for simulation tests.

Usage: \texttt{execute\_test.sh [-h|--help] [-c|--compile] [-n|--no-gui]}

\texttt{[-a|--asm] [-f|--freertos]}

The option \texttt{-h} or \texttt{--help} displays detailed usage information, \texttt{-c} or \texttt{--compile} recompiles the targets before running them in a simulation, \texttt{-n} or \texttt{--no-gui} runs the simulation headless, \texttt{-a} or \texttt{--asm} uses tests in \texttt{miriv/software/asm} and \texttt{-f} or \texttt{--freertos} uses FreeRTOS as a target.

It is recommended to increase the tick interrupt frequency setting for FreeRTOS or reduce the clock frequency such that timer interrupts can be made visible early in the simulation.

To use it, navigate to the directory and launch it:

\texttt{cd miriv/sim}

\texttt{./execute\_test.sh}

\end{document}