This is the page for the "Hardware and Software Codesign" course at Università di Pavia, academic year 2021-2022. It started in March 2022.
This tree, excluding the big mp4 files, can be cloned on your
system; then you could "git pull" for updates. The MP4
files are not committed to avoid wasting too much disk space on your
laptop (once in the database, once in the checkout).
You can still download the video files separately if you want.
To get a local copy of the web pages:
git clone git://hsw2022.gnudd.com/hsw2022-www.git
The "main" course tree, that includes both a "hw" subdirectory and a "sw" one, can be cloned like this:
git clone git://hsw2022.gnudd.com/hsw2022.git
Both repositories grow during the course.
Exercises live in a separate file called exercises.html.
Everything is in this single file (I hate fragmented pages). But here is a table of contents (one entry per lesson) as a shortcut.
--gc-sections and stackThe material is organized in chronological order. Each 2-hours lesson has a sequential number, starting from 01. The introduction is called 00, even if lesson 0 does not exist. When something is missing, a FIXME appears.
All lessons are recorded, unless recording failed for "technical" reasons (which include human errors). Recording is available some time after the lesson, because I need to recompress and upload. Usually this happens on the same day, but sometimes I'm too busy with work to do it immediately.
Sometimes there are short videos, usually from previous editions of the course. If I link here previous videos, it means they are still relevant. You are welcome to look at hsw2021 or hsw2020, but the subject matter may be different. As a general rule, hsw2020 has more videos (all the teaching was remote in 2020), but the first one are at awful quality, because I took a while to find the right tools to record.
The videos are 15-20m each. Specific length is listed here below for each of them. They are on Youtube, feel free to share the link, because I (almost) don't swear when I record myself. As an alternative, you can download the mp4 file. If you watch online please ensure to make the image big enough. For the mp4 version (which is the original I uploaded, so it is either the same or better quality) I verified that what I write is readable. Please remember that most of what I show is committed as well.
This is an introduction to the course, showing some hardware devices and giving an idea of what we a going to learn to do. The video is from hsw2021.
We are using git all the time (and you need it elsewhere too). Most of you are already acquainted with basic use of the tool, but advanced features, which we'll use routinely, are unknown to many. This is a description of why and how it is done its way, while showing some commands in real repositories.
If you are new to git, please do not be scared, getting proficient takes time and experience. Please try to get the basic ideas out. If you are already used to it, I hope you can get something new from this lesson.
The videos are from hsw2021, and I feel them current. the slides I used in the lesson are slightly different.
Some questions and study hints are here.
This talk by Linus Torvalds is very interesting, despite its age. As everything else, especially my lessons, 2x playback is your friend.
Advanced Git Use is an article I published in 2010 but it's still current, because the basic ideas are unchanged. I translated it from Italian in a hurry, and later tried to fix the disgusting English, but I still beg your pardon for the bad language and typos. Time doesn't allow a fix. (patches are welcome). I also removed a section I don't find relevant any more.
The man pages gittutorial(7) and gittutorial-2(7) are
a good starting point if you look for a tutorial. The man
page git(1) and all pages for the sub-commands
(e.g. git-log(1)) are very good reference material. Please
note that sub-comamnds are separated by a space, but man pages use
the dash, because the argument of man must be a single
word.
If you miss local man pages, ore prefer a web render of the very same information, you can use the following ones; but please remember that the local man refers to exactly the version you have installed, while the web describes the "current" one, whatever it means -- not a problem for git, which has a stable interface, but a real issue for some projects.
In this lesson we cover C language. Every attendee already knows C, but what we are interested in is the use in the microcontroller world.
We start with the normal hello.c that uses the standard
library to then show how to not use the standard library (like
we'll do in embedded systems later). Finally, we introduce
a trivial linker script to show how you can build an array from
items that live in separate source files. We'll do more linker script
later: it's an important part in building a firmware image.
I find the three support videos of 2021 are still current. The topic was covered in lessons 3 and 4, and file are named accordingly. The first one shows how you can avoid using the standard library, with some details about how hello.c interacts with the rest of the world; the second one introduces povacca and use of a data structure with function pointers; the third introduces the linker script and ELF sections.
Related code is in the hsw2021 repository. As usual,
all steps are committed, you are expected to look at the history and
not what is in master, which is just the last version. When
I refer to specific commits, they refer to that repository.
The lesson is recorded. Students got the reference, but I published
no reachable link. The code is committed in the hsw2022 repository.
The following references are not new, nor specific to this course, but still useful, it's stuff I wrote ages ago when I first had to teach embedded programming. The operator table is just a reference about priority of the operators (but extra pairs of parenthesis are always accepted), but the @i{A-C-X} articles are something I'd urge to at least to open and look around before we go to the next lesson.
Some questions and study hints are here.
Makefile?if/elseif?__attribute__((section(".oper"))).Today I'm showing how to create a VPS (a virtual private server).
During the lesson I'm buying a VPS from a specific vendor, and the
server will stay online for 6 months. All students are (strongly)
invited to have an account there, where they can create repositories
and build stuff (there are no limits on resource use. This server
is then associated to the name
stu22.gnudd.com by filling a DNS entry.
In order to create an account, each student must send me an ssh public key. This can be generated by ssh-keygen if you run an operating system or by puttygen if you run windows. The public key is a single, long, line, like the following (the line here is split for typographic convenience):
ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCxQJou6IEVLP5zOpWdc3UkQX4bt44f517TE YqPprmobHtXO7doIMHjfPQME1zW3iACAHGNikNbg8ox6yiN5r1YA8PIuDXNF8pkwXv7Str23y /MeRg9kCmt77i+xN2ydaTyWk3bkgOXh4roIj5QE1RS4sTT3QlONZezaIaoAAdQC4UpG9uwAsB U10tb8tNFtWAfXtGPa3HSA+308ZKROVglQ1S96hA7i88okuf5z3hRJaWyE7meZ4RbDmgpttRH 8ktdUYex2jmCC/ARCeFRG4yePWWZnTd7znitiYBl+pGNHlKXjSRuxd5i81rdgioqyZS15lnBb yuXVZ+iuDPAdEHL rubini@morgana
You are expected to send me this public key by email, so I can create a user in the remote host and save this key as authorized. The ssh (or putty) client will prove your identity by using the private key (which remains withing your computer) -- I don't remember the details, which are not important, but this is most likely a challenge-response kind of exchange.
Then, we talk about PCB design. But this is not a tutorial about practical use of a specific tool, but more a collection of ideas and suggestions. We may have a tutorial lesson later of, if the audience requests that.
You are expected to install Kicad (which is Free Software) and get acquainted with it, at least slightly. You may want to design something for the project work you'll deliver at the end of the course. If you prefer a different tool, that you can legally use, that's not a problem, but please consider whether you'll be legally allowed to use it again in the future.
The lesson is recorded, as usual, but you may want to rather look at the short videos from last year. Please note that I was using an ancient version of Kicad, so some practical details when I open my windows are different, but the basic ideas and suggestions are the same. Video file names are "02" and not "03" because I made them for lesson 2 in hsw2021.
Video "02-1" is a talk based on the slides, with some reference to the hardware device, made by opening Kicad and showing the basic flow of the tool. But the Kicad version used here is old, and some things are now different (and simpler, in a way).
Video "02-2" shows my workflow and a number of examples. It may be interesting to look at, because I show a few devices and the design ideas that each includes.
Video "02-3" covers the last few slides, where I list my own suggestions for circuit designs and explain the reasoning behind them. The subject matter is unchanged this year.
The slides for this year are slightly different (updated) from last year's set:
And tdc.pdf is the schematic of the board we'll use later on.
Finally, Italian-speaking (or, better, Italian-listening) readers may enjoy a presentation of TDC-1.0 I made at the End Summer Camp 2019. It is a shorter version of a 2-hour conference session where I had my oscilloscope and everything to actually show the board in action.
Version 1.2 of TDC (that I designed in March 2022) includes more peripherals than what in in that video, but it is derived from the one shown there. In git history you see the changes from v1.1 (and in last year's repo you see changes from v1.0, which I didn't preserve in the new history).
Some questions and study hints are here.
Today we got back to the povacca program, looking a little
more in detail into attribute, static and other details
of freestanding programs, in relation with compiler optimizations.
We thus added attribute(used) in povacca.h.
We also looked at the VPS that was created in the previous lesson,
and the steps needed to create a user, allow login with a public key
and set name and email in the git configuration file. We
also added stu22 to a DNS domain, in order to be able
to use a symbolic name for the IP address.
I covered a set of slides about the development system: basic ideas
about the role of the linker and the fundamental syntax of
the Makefile. The slides cover everything that we'll
ever use of make: there is (unfortunately) a lot more, but we
don't need to ever use it. Also, all the "strange" things introduced
here will be explained again when we use them in practice.
The last slide introduces the syntax for inline assembly, which is not
trivial at all. Again, it will be clearer later when we use it in practice.
Finally, this lesson changes the examples/02-hello
directory of the repository to add a cross-compiled version
of hell (which is hello without a standard
library). And we found that, strangely, the hell binary
can be run on the PC even if it is ARM binary code.
__attribute__((used)) mean with regard to optimizations?This lesson covered the topic of cross-compiling, for which I have a slide set. Again, I'm interested in the ideas, not the details.
Then, we started to build a freestanding application that runs on the versatile development system. We use qemu-system-arm to simulate that very board and run the code. As usual, in the repository you find the complete history (with verbose log messages where useful) from the empty file-set to the current status. What you find in master at any point in time is not really important, what's important is the path that leads there.
CROSS_COMPILE in the Makefile?vello?static a function?This lesson completes support for a "real" hello world application running on the Versatile hardware (as faked by qemu-system-arm).
The point is that the previous vello implementation had a lot
if hidden magic stuff, so we are now uncovering it. We introduce
/DISCARD/ in the linker script, a boot.S inizialization
file and the idea of jiffies as an hw-updated time counter.
Finally, we use regs[] as a way to access all hardware
registers, in a way that makes it clear, to the code reader, what
operations are hardware accesses, and what operations are not.
By relying on jiffies, we end up with a new version of vello.c that prints its message once per second. So we put in place everything to run a firmware image on a microcontroller, even if it is now running on emulated hardware.
/DISCARD/ in the linker script?regs[] to access machine registers?regs in the linker script?volatile mean?After running on emulated hardare, we are now moving to real one. Each student is expected to have a specimen of the board, in order to test what we are going to write and hopefully hack own code.
The TDC kit is described in the next section.
In this lesson (please see the commit log for all information)
we introduced a complete linker script, with a few more features than
what we used in the versatile case. In particulare, we define
memory areas (MEMORY { ... };), because a microcontroller
has both flash and RAM memory, and in the end code will end up in flash.
Please note that we allocate the stack at the beginning of RAM.
Then, we add the definition of some machine registers, taking the names
and addresses from the user manual of the target processor. Registers
are a boring part of the work, but we need to name them (e.g: see
commit a48ced38). And boot.S, putc and puts,
very similar to the versatile ones.
Finally, a Makefile is added. This is longer than expected
because it's already laid out for the final version of the code base.
Despite the length, nothing strange is there: it's a list of dependency
rules, and the only "strange" syntax is limited to the few features
described in the slide set of lesson 4.
tdc-11-test as a microcontroller progra?$(wildcard) in the Makefile?Section to be filled. Pleae refer to last year's page by now, but please remember that we are not performing serial programming, in order to keep things simple. We'll just copy binaries to the internal flash of the microcontroller.
This lesson covers some information about ARM processors and their assembly language, explaining where Thumb-2 comes from.
It then covers GPIO pins and offers a possible API to configure and
access such pins. The students are urged to think about a better API
than what is shown here. The current uglyness is apparent when
looking at tdc-11-test.c as committed.
gpio_dir_af() API a bad design choice?THRE to be empty?This completes the discussion of gpio pins, by actually commiting the code and showing the typical led-toggling program, by using the provided gpio API.
Then, we introduce programming to flash. This means setting up the UART (because when the microcontroller leaves reset state nothing is setup. The uart-programming step is not needed when we run code in RAM (because the internal boot-loader configured it for us), but this year we'll skip this ram-programming environment, to keep things simple.
The UART adapter is needed anyways, to collect output strings (and later to send input strings), and we are not going to introduce USB code (interested readers can find it in the 2020 lessons).
ARRAY_SIZE a useful macro?.text.ramboot and .text.romboot.rep and .repr and why are they used in udelay()?Today we introduced the Neopixel (WS2812 and similar devices) and their peculiar protocol. Three of them are mounted on the TDC device. We show a nop-based implementation, with its pros and cons.
The other big topic is Kconfig. We are not interested at all in the
Kconfig code, just like we don't dissect the compiler or editor, but
we are dealing with the way configuration is accomplished, and the
trick of obj-y. This originates from the Linux kernel
(whence the 'K' of Kconfig) and is now used by a number of important
free software projects. Other environments copied the basic idea,
as this approach is much simpler than any alternatives ever conceived.
As usual, the commit messages include the important bits. Sometimes the code is important as well, and sometimes it is not (as a general rule, short commits are important and the ones with a lot of code are not, because nobody is interested in the boring details).
No questions are provided, because nobody answered any of them so far. Some are asked in the classroom, if I feel interest in the audience. Recordings are available at the usual link.
nops in commit 3507a05?obj-y and LDS-y in Makefile.This lesson covered the concept of PLL and the code to raise the uC
frequency. We can only multiply by an integer number and the maximum
declared frequency is 50MHz. With a 12MHz oscillator we can only use
values 1..4, which are already offered by Kconfig-lpc
As usual, we are not interested in the details of the code or specific
registers. What is important to note, however, is that there is a
FLASHTIM register that deals with flash wait states (i.e.
"flash timing", whence the name. The default setup for this register
is "as slow as possible", so no mishaps happen when we turn on the PLL,
but we might (and should) lower the number of wait states when running
at lower frequencies. In the microcontroller world, where most applications
are power constrained, it is actually pretty common to not run the processor
at its faster allowed speed.
The last commits deal with the performance problem we had with neopixels,
where we had to unroll gpio_set inside neopixel.c
itself, to avoid the overhead of a function call, which completely
disrupts our timing. To make the code cleaner and more portable,
we introduce __gpio_set() and __gpio_get() as
static inline functions. Thus, no function call is
emitted by the compiler and setting (or reading) a pin just takes
two CPU clock cycles. With this in place, neopixel.c
is fixed accordingly.
Please note that the current neopixel code is still bugged, because it only works at 12MHz (and assumes we have 3 devices on-board).
__gpio_set() API and why we use __ in the name?This lesson covered just a few commits about timing. Part of the time was spent looking at bad practices for gpio/led access, referring to published examples of other build environments.
The important things, as seen in the commits ("git log
--reverse lesson-11..lesson-12"), are the idea that the HZ choice
can be folded in Kconfig, that you can shortcut functions based on
build-time constants, and that you can avoid the bug of jiffies overflow
using proper comparison functions -- here time_before() and
friends.
CONFIG_PLL_CPU is 1?HZ to be a build parameter rather than a constans?--gc-sections and stackSo, eventually, we add printf. The implementation is picked from another project of mine, called pp_printf, that I wrote ages ago for a client, starting from an old linux-kernel implementation.
The plus of pp_printf is that we developed for a client's tool), and is configurable: you can choose at build time which of several versions to run, choosing your (application-specific) trade-off between size and features. Any printf format string will achieve the expected result -- in other words, the smaller versions do not mess up with format specifiers, they just don't obey some formats.
As usual, the first commit in the printf branch picks a commit from upstream, so you can track your code source. Then the code is fixed to actually build and Makefile is modified last, so each and every commit builds without error, to avoid issue if you need to git bisect in the future.
The commit set also introduces PROVIDE in the linker script.
Another feature introduced in this lesson is the --gc-sections
linker instruction. This is used to discard, at link time, any code or data
that is not used by the actual application. All details are in the
commit logs, including the KEEP directive in the linker script.
More details are in the ld user and developer's manual.
The final set of commmits shows how to pre-fill the stack with a magic word, in order to detect the application's use of the stack space. But it's nothing magic, just a loop at init time, that takes a few microseconds and has no effect at run time.
Based on this, I introduce a simple application that forces a stack overflow, while monitoring stack use. As expected, it eventually hangs, when accessing invalid memory at bottom-of-stack.
--gc-sections linker option?KEEP in the linker script?Before the timining, this lesson introduced Reflection on Trusting Trust by Ken Thompson. A mandatory read for my students, because I fear it was not yet introcuded in earlier courses. It's simple and enlighting, in my opinion. It's old, but great ideas never rust.
Then the main subject matter is a new udelay implementation.
The important idea is autodetection at run time: a small probe overhead
makes the code robust for real-world environments (as opposed to
#idef CLOCK_8MHZ and similar perversions). The other important
tool, already known, is use of constant expressions (especially those that
are constant at build time).
The code as introduced as a bug, that we later solve. A test program helps in detecting the bug, which is clearly visible through the usual tools in binutils.
Finally, we fixed the neopixel code, in order to work with any CPU frequency.
The tool used here is .rep/.endr, which was already the base
for the previous udelay implementation. Based on the fact that there's
a lot of margin in the data-low timing, we can easily achieve a reliable
neopixel data stream. Obviously this assumes there are no interrupts.
__attribute__((noinline)) and why do we use it?This lesson discusses about real-time scheduling, introduces the most important schedulers (RM and EDF) and offers an implementation of the "task" idea that relies on a single stack (but we still have no preeemption, and this year we'll likely won't cover that part).
With this, we eventually managed to cover all of the topics of my "two hours operating system" talk, so here is a link to the thos material, which should be a quick recap of everything we have seen so far. The recorded video is 90 minutes, it was taken at a conference.
__task_first and __task_last in the linker script?struct task?Today we merged a sscanf implementation, and a command mechanism. This requires a way for async input, because we can't stop all operations while waiting for serial input. Thus, pollc and polls are committed first.
The polls function reads the serial port while accumulating
a string, and returns non-null only when it sees a string terminator.
Thus, a generic polls_t, functin is implemented, that returns
a string when any provided terminator is seen; so polls is just
a call to polls_t with "\r\n" as terminator list.
Then, <errno.h> is introduced, so pollc
can return -EAGAIN in case no byte is there. Such
constant is a standard POSIX error code.
Finally, we create a timestamping function that uses both the jiffies counter and the intermediate fraction to return a clock-cycle-sharp timestamp. Obviously, timestampit itself takes time. The implementation allowed to discuss about the problems in reading a dual-word value, as well as inline assembly, which we used in order to be faster in the timestamping code.
errno.h used for?In this lesson I described the most important communication busses: I2C, SPI and UART (including the RS-485 variant). It's important to note how most users wire the RS485 tranceiver in the wrong way, while it's to the next lesson.
In this lesson we introduced code (written live in the classroom) to measure the worst case execution time of all tasks.
As usual, the implementation is based mostly on a data structure. Here we add fields to the task strcture, so we can track the WCET of each task. Actually, we may remove some fields using local (temporary) variables, but that won't work when we introduce preemption. In general, it's better to leave all data traceable (with a debugger or otherwise), unless the size overhead is important for the specific applications.
No exam questions.
Today I covered the "strange" way data strctures are done in the
kernel. Based on conainer_of you can do great things, and we
looked at linkerd lists and red-black trees. Examples using the
kernel code in user space are committed
as example/19-datastruct.
container_of doing?list.h.In this lesson I introduce the concept of initcalls, an idea
widely used in Linux. The point is that the initialization function
of any "module" should be automatically called at initialization time
(here: setup.c) whenever the module is used.
Unfortunately, the current code organization, where we build a big
object file with all modules in (to
later --gc-sections) shows it limitations. Thus, the
first set of commits is changing how the code is set up: we build
library (called libfsmos.a) and link with that library.
In this new library-based approach, if the application refers to a
"module" (a source file, which can be a driver or a software
feature), the corresponding object file is linked in the final
binary; if no reference is there, nothing is picked. Initialization
code is never directly referenced, because it is used in a specific
ELF section with the usual trick of first and last;
but being marked as KEEP it is not discarded. But if a
module is not referenced, the respective object file is never part
of the link and its initialization function will not execute.
Finally, init.h and the linker script take care
of the internal details about using an ELF section.
This lessons puts to practice some of the ideas shown earlier.
The commits in the interval lesson-20..lesson-21 were
written in the classroom.
What we did is changing the tests-lpc/r-w-go
shell-lookalike application into a task, that can be scheduled by
the simple cooperative scheduler we already have. We made it an
optional task automatically picked by the scheduler whenever the
scheduler itself is used; finally we removed r-w-go.c
to add shell instead, which is a one-liner.
Then, we added a wcet command to the shell, to print
the worst case execution time of all tasks (as collected so far, when
the command is invoked). This is turning a quick hack (wcet calculation)
into a system feature, easily activated by every application.
And, not least, having a system feature means that the hack can be
removed, so manual printing of wcet in the tasks example is just
reverted (git revert <commit^gt;).
Finally, we added another "system" feature: the sleep command.
This can be useful to inject unexpected delay in the application during
testing. Actually, we used it to see that the sched_errors
mechanism is actually working, and that an overload in one task (the
shell) does not disrupt further correct exaction of periodic tasks.
The point of this all is that if you lay out your code features properly (i.e., based on data structures using an object-oriented approach), you can avoid a lot of code duplication later and easily add diagnostic tools that can quickly be turned into system features, available to any application on request (e.g. Kconfig, so you can still remove them if code size in production is an issue).
No exam questions.
The exam is greatly simplified, if compared with previous years.
You will be asked 10 questions, and you must reply correctly to 8 of them in 90 minutes. The questions are chosen from the ones written above at the end of each lesson.
The exam is done with pen and paper, in the classroom. This is why students are given 10 minutes per answer, plus 10 minutes overhead.
You can use the laptop, to look at commits and whatever you chose to have locally, but without any network connection.
In the questions, you are expected to describe with your own words one technical point. Answers must be as tidy and to the point as possible (i.e., no hand-waving). You choose your questions, you must know the answer.
Technically, you can cheat, because there is no police in the classroom. However, cheating is not accepted. If your deliverables are suspect, we may agree to have a short interview. I really hope this is not needed.