It is often said that you do not know something for real until you have taught it to someone else. Recently, I had the delightful experience of working through such an experience with Fernando Molina, an intern at Intel in Costa Rica. We worked together to create a workshop for university teachers and researchers to show them how they can use Wind River® Simics® tools and virtual platforms at universities and other institutions of higher education. Fernando ended up with one of his university teachers in the classroom – teaching his own teacher! During the process of developing the workshop, Fernando also had to learn Simics and apply it to the task of developing a Linux* device driver and associated test programs. Lots of learning on all sides.
The workshop was based on a complete system stack simulated in Simics: Generic Intel® PC-style system model, PCI-express* (PCIe) device, device driver, Linux kernel PCIe system, and application-level software using the device through several Linux operating system (OS) application programming interfaces (APIs). We created a simulated PCIe device that featured buttons for input and a few light-emitting diodes (LEDs) for output. In the real world, this would have been something like a panel attached to a PCIe slot in a PC – not a very realistic device in terms of functionality, but a great basis for explaining how devices and device drivers work.
The setup is shown in figure below:
JE: Fernando, thank you for joining me for this interview. Could you please start by introducing yourself to our readers?
FM: I am Fernando Molina, from Costa Rica. I’m a Computer Science student at the Costa Rica Institute of Technology (also known as Tecnológico de Costa Rica, TEC) on my last semester. I started working at Intel in August 2016 as an Intern. This is my first job and I’m very happy to start at a place like Intel.
JE: What did you do in this project?
FM: I supported the idea to have a workshop to show Simics capabilities to academia in my country. Simics is a powerful tool that can be used as a learning aid in classrooms, and the idea was to create a setup that could explain teachers through example what Simics is and what it is capable of doing. During this project, I had to work with the Linux kernel to create some procedures in a device driver for a small LED board that was connected to the simulation through PCIe. This was a challenge for me, because of my background I had never worked with Linux in such a low level, but the Simics tools also helped me to get into the right track fast. I mainly worked using C to write the driver and then Simics to test and debug it.
JE: What did you learn about Simics?
FM: I learned it is an incredible tool for software development, especially for low level software. Simics provides a lot of information that can be hard to get using normal software development tools or normal hardware debugging approaches. Changing the code, building and testing a new version didn’t take more than a minute, which made the development really fast.
JE: How did that work?
FM: The magic of Simics automation. Using Simics, it’s a matter of seconds rebuilding the driver code, loading a previous checkpoint with the machine booted and install the new kernel module from this, without having to wait for the platform to boot from scratch! This made development very fast, just a matter of recompiling and launching the new Simics from the checkpoint.
Here is the part of the script that does the loading and installing the kernel modulo ($ insmod) of the kernel driver (after we started from the checkpoint):
JE: In the driver updates, how did you deal with problems that arose?
FM: The Simics Eclipse tools help for debugging and pinpoint errors more easily. You can even debug Kernel code if you like! I found out Simics is a perfect tool to develop your software for device or platforms that are not physically available, and even if they were, development (and specially debugging) using Simics is much more convenient. Repeating problems was very easy.
JE: What did you learn about PCIe and Linux?
FM: I learned a lot about PCIe, given that I had no prior experience with it. PCIe and Linux work nicely together, the Linux kernel maps the device automatically to an available address and it then can be accessed through file handlers or direct memory accesses. Your help was appreciated in this topic.
Here is an example of the code of the device, showing the start of the PCIe configuration space of the model. This code is written using Simics DML, a domain specific-language for device modeling that is used to quickly build device models for Simics:
The template for the register bank provides reasonable defaults for all the mandatory registers, so the declaration here just provides the specifics that makes this device different. In particular, the various IDs required by the PCI standard (and its PCIe successor) to identify the device so that the correct driver can be loaded.
Following that, there is setting of the status.c bit. This indicates that the device has PCI capabilities defined. The capabilities are used to declare that the device uses extended message-signaled interrupts (MSI-X interrupts), among other things. The bit was missing in the first drafts of the device model, and as a result the Linux kernel refused to set up IRQs and send interrupts. After some following of the calls in the kernel and looking at the replies from PCIe APIs, as well as a second reading of the PCIe standard, it was clear it was needed. With a virtual platform, that kind of exploration from scratch is possible in a way that is really difficult to do in hardware.
After that, the code visible here sets up the base address registers (BARs) that make the programming registers and MSI-X interrupts banks addressable on the processor’s memory bus.
JE: I remember my own experience building my first device driver a decade ago for a previous project in the same vein. But back then, it was just a plain memory-mapped device and a 2.6-series kernel. PCIe is rather different… you get a lot more help from Linux, but there are also more things to do right.
FM: Yes, of course. I found PCIe to be a very complete (if not complicated) standard. In the model, I could see a lot of registers that the standard defines and these registers work nicely together with the Kernel to help map the device into the PCIe hierarchy. Luckily I didn’t have to make anything with those registers, the base driver you gave me at the start already had the Kernel connection part worked out, but I had to program the memory-mapping mmap() device driver API calls and the device’s file descriptor read/write functions.
JE: How was it to work with device drivers on Simics?
FM: Once I understood the driver basics (not taking into account the PCIe connection part), it was pretty straight-forward. Since the device driver execution is made in a Kernel execution context, one needs to be careful with the way one codes because it is really easy to make a mistake, but since the device driver we were programming would basically just blink some LEDs in a panel there wasn’t really much room for errors.
JE: Let’s take a slightly deeper look at what we used in the workshop. Here is a screenshot of the setup running (on top of a Fedora* Linux):
The user enters “echo” commands on the target system serial console. These echo commands send characters to the device node for our device driver, which parses the provided strings and then lights up (or turns off) the LEDs in the System Panel as appropriate. In the black “Textual Graphics Console”, we see the diagnostic output from the admittedly rather chatty device driver (it is intended as a teaching tool, after all).
This shows the path from user applications through the filesystem to the kernel to the device driver to the PCIe memory mapping to the device and finally to the output in the system panel. A complete pass through the stack – all driven by a plain “echo” command.
JE: In your opinion, what did the participants think of the workshop?
FM: I believe the participants discovered a tool I’m quite sure they had never seen before, at least here in my country. Simics is an impressive full system simulator, and I’m certain the participants now know this as well. In an academic environment, Simics can be used for subjects like Embedded Systems, Operative System or Computer Architecture. The simulator can help students to understand the concepts taught in the classroom more easily with hands-on experience.
JE: Which parts were the most interesting to the participants?
FM: I think the most interesting part in the workshop, at least for me is watching Simics debugging capabilities. It is impressive how easy it is to dive into the device driver’s source code for inspection and from there, even being able to step through operative system routines was quite amazing the first time I tried it and I’m sure the participants felt the same way. Also, being able to communicate two simulations through network is interesting as well, and demonstrate just hoe powerful Simics can be for hardware and software development.
JE: We also had an interesting time getting the materials packaged in an easy-to-use way. USB keys sounded like the best idea… but then we got into some trouble.
FM:Ha ha, yeah.
We found out the hard way that not all USB keys (even if they are USB 3.0) have the same read/write speeds. We first tested making a bootable USB key (this means, the USB key behaves like an external hard drive and the user boots the entire OS from the USB key as the main storage drive). This worked like a charm for the USB key I used in development.
We then proceeded to copy this image into newly bought USB keys from a different manufacturer. When we tested these new keys, the performance was very poor. Using the initial USB key, the OS would behave like a normal PC, but with the second type of USB keys, the booting took three or four times as much and the general performance was unusable.
When I benchmarked the USB keys I found out that while both USB keys had read speeds close to 100 MB/s, the initial USB key had a write speed of 50 MB/s whereas the second set only just reached 15 MB/s! This made it unusable to use as a bootable USB key, since the boot writes quite a bit of data to the disk. It also affected the performance of Simics Eclipse, where some operations would bog down completely due to being write-intensive.
The moral of the story is, if you test something in one environment, always make sure the environment is the same if you want to get the same results.
JE: Thank you very much!
Closing Remarks
This work shows how Simics can be used as a teaching tool, at all levels of the software and hardware stack down to the internal logic of a hardware design. It does not cover RTL (Register Transfer Level) and implementing the design in actual hardware, but it provides a good way to show how things fit together.
In my opinion, all students that will work with programming, or computers, or system design in some way should have a basic understanding of the whole software stack. When balancing on top of the stack, it is good to know what is in it. When writing code down at the bottom, understanding what goes on at higher levels is definitely important. When designing hardware, understanding what drivers find useful is very important. Understanding the whole software stack from the hardware interface to the application software is important. And Simics is a great way to get that understanding, with the ability to work at any level.
For more such intel IoT resources and tools from Intel, please visit the Intel® Developer Zone
https://software.intel.com/en-us/blogs/2017/12/05/windriver-simics-to-inspire-teachers-costarica
Follow Us
