New Horizons

Welcome to my blog

My name is Sven Andersson and I
work as a consultant in embedded
system design, implemented in ASIC
and FPGA.
In my spare time I write this blog
and I hope it will inspire others to
learn more about this fantastic field.
I live in Stockholm Sweden and have
my own company


You are welcome to contact me
and ask questions or make comments
about my blog.


New Horizons
What's new
Starting a blog
Writing a blog
Using an RSS reader

Zynq Design From Scratch
Started February 2014
1 Introduction
Changes and updates
2 Zynq-7000 All Programmable SoC
3 ZedBoard and other boards
4 Computer platform and VirtualBox
5 Installing Ubuntu
6 Fixing Ubuntu
7 Installing Vivado
8 Starting Vivado
9 Using Vivado
10 Lab 1. Create a Zynq project
11 Lab 1. Build a hardware platform
12 Lab 1. Create a software application
13 Lab 1. Connect to ZedBoard
14 Lab 1. Run a software application
15 Lab 1. Benchmarking ARM Cortex-A9
16 Lab 2. Adding a GPIO peripheral
17 Lab 2. Create a custom HDL module
18 Lab 2. Connect package pins and implement
19 Lab 2. Create a software application and configure the PL
20 Lab 2. Debugging a software application
21 Running Linux from SD card
22 Installing PetaLinux
23 Booting PetaLinux
24 Connect to ZedBoad via ethernet
25 Rebuilding the PetaLinux kernel image
26 Running a DHCP server on the host
27 Running a TFTP server on the host
28 PetaLinux boot via U-boot
29 PetaLinux application development
30 Fixing the host computer
31 Running NFS servers
32 VirtualBox seamless mode
33 Mounting guest file system using sshfs
34 PetaLinux. Setting up a web server
35 PetaLinux. Using cgi scripts
36 PetaLinux. Web enabled application
37 Convert from VirtualBox to VMware
38 Running Linaro Ubuntu on ZedBoard
39 Running Android on ZedBoard
40 Lab2. Booting from SD card and SPI flash
41 Lab2. PetaLinux board bringup
42 Lab2. Writing userspace IO device driver
43 Lab2. Hardware debugging
44 MicroZed quick start
45 Installing Vivado 2014.1
46 Lab3. Adding push buttons to our Zynq system
47 Lab3. Adding an interrupt service routine
48 Installing Ubuntu 14.04
49 Installing Vivado and Petalinux 2014.2
50 Using Vivado 2014.2
51 Upgrading to Ubuntu 14.04
52 Using Petalinux 2014.2
53 Booting from SD card and SPI flash
54 Booting Petalinux 2014.2 from SD card
55 Booting Petalinux 2014.2 from SPI flash
56 Installing Vivado 2014.3

Chipotle Verification System

EE Times Retrospective Series
It all started more than 40 years ago
My first job as an electrical engineer
The Memory (R)evolution
The Microprocessor (R)evolution

Four soft-core processors
Started January 2012
Table of contents
OpenRISC 1200
Nios II

Using the Spartan-6 LX9 MicroBoard
Started August 2011
Table of contents
Problems, fixes and solutions

FPGA Design From Scratch
Started December 2006
Table of contents
Acronyms and abbreviations

Actel FPGA design
Designing with an Actel FPGA. Part 1
Designing with an Actel FPGA. Part 2
Designing with an Actel FPGA. Part 3
Designing with an Actel FPGA. Part 4
Designing with an Actel FPGA. Part 5

A hardware designer's best friend
Zoo Design Platform

Installing Cobra Command Tool
A processor benchmark

Porting a Unix program to Mac OS X
Fixing a HyperTerminal in Mac OS X
A dream come true

Stockholm by bike

The New York City Marathon

Kittelfjall Lappland

Tour skating in Sweden and around the world
Wild skating
Tour day
Safety equipment
A look at the equipment you need
Skate maintenance
Books, photos, films and videos
Weather forecasts

38000 feet above see level
A trip to Spain
Florida the sunshine state

Photo Albums
Seaside Florida
Ronda Spain
Sevilla Spain
Cordoba Spain
Alhambra Spain
Kittelfjäll Lapland
Landsort Art Walk
Skating on thin ice

100 Power Tips for FPGA Designers

Adventures in ASIC
Computer History Museum
Design & Reuse
d9 Tech Blog
EDA Cafe
EDA DesignLine
Eli's tech Blog
FPGA Arcade
FPGA Central
FPGA developer
FPGA Journal
FPGA World
Lesley Shannon Courses
Mac 2 Ubuntu
Programmable Logic DesignLine
World of ASIC

If you want to be updated on this weblog Enter your email here:

rss feed

Thursday, June 05, 2014
Zynq design from scratch. Part 48.
Measuring Linux usage

I know it is impossible to measure and figure out the real usage of different Linux distributions, but there is data collected from the web that can give us some clues. RedMonk has put together the following graph taking data from different sources.

After studying these figures and looking around in my neighbourhood I think one thing is clear. Debian together with its derivatives (Ubuntu and Mint) is by far the most popular Linux distribution in the world.

Xilinx Linux support

So why is Xilinx not supporting Debian? I don't know. But this doesn't stop us from using Ubuntu to run Xilinx software. I have been doing it since 2007 when I started using Xilinx design tools and continued up until today. So let's install Ubuntu 14.04.

Installing Ubuntu 14.04 LTS

The latest version of the Ubuntu operating system for desktop PCs and laptops, Ubuntu 14.04 LTS comes with five years of security and maintenance updates, guaranteed.

Follow the instructions in part 5 to install Ubuntu.
Yes it is that easy.

Fixing Ubuntu

Folllow the instructions in part 6 to perform the necessary fixes to make the Vivado run smoothly.

Installing Vivado

Follow the instructions in part 7 and part 45 to install Vivado and SDK and you are up a running in no time at all.

Top  Previous  Next

Posted at 09:36 by svenand
Make a comment  

Sunday, May 18, 2014
Zynq design from scratch. Part 47.
LED_Pushbuttons application program

Starting Xilinx SDK

--> xsdk &

Re-generate BSP sources

After we have modified our hardware design and added the interrupt logic we need to re-generate the Board Support Package to incorporate support for interrupts. Open the system.mss file, if not already opened and click Re-generate BSP Sources.

Create a new application program

We will keep the LED_Dimmer application as is and create a new application called LED_Pushbuttons.

Writing the program

We will use the LED_Dimmer c-program as a starting point and add support for push buttons and add an Interrupt Service Routine (ISR).

Not reinventing the wheel

Let's find some good examples that can help us implement the interrupt handling in our application. Open system.mss and click on examples for the axi_gpio_0 peripheral.

We will take a closer look at the xgpio_intr_example.c file.

My implementation

This is what I came up with. Take a look. Let's compile and run the program.  See part 19 for more information.

Here is the result.

Top   Previous   Next

Posted at 16:18 by svenand
Comments (3)  

Saturday, May 17, 2014
Zynq design from scratch. Part 46.
Adding push buttons and interrupts

This tutorial demonstrates how to modify our Zynq system from lab2 by adding inputs to the custom PL peripheral and connecting them to PS interrupts. This lab will show how to:

  • Modify the project's existing PS and PL subsystems
  • Update the C application for new hardware
  • Add interrupt handler and interrupt service routine (ISR)
  • Download and test the software application in hardware
1. We will use Vivado 2014.1 to build our new system.

--> vivado &

2. Open the LED_Controller project.

3. Expand system_wrapper in the Sources pane and double-click system_I -system ( to open Block Design. This is the new 2014.1 look.

4. Start to configure the AXI GPIO block by double-clicking the IP.

5. The Re-Customize IP window opens showing the AXI GPIO. Select the IP Configuration tab and check the setting for Enable Dual Channel. Set the GPIO 2 to All Inputs with GPIO Width 5. Check the setting for Enable Interrupt as well. Click OK.

6. Start to configure the PS block by double-clicking ZYNQ7 Processing System.

7. The Re-Customize IP window opens showing the ZYNQ Block Design.

8. We will start by adding a low speed clock (FCLK_CLK1) running at 250KHz and clocking the debounce circuit. Select Clock Configuration from the Page Navigator.

9. Before we can connect the interrupt signal we need to enable interrupts in the PS. Select Interrupts in the Page Navigator pane to the left and expand Fabric Interrupts and PL-PS Interrupt Ports. Enable Fabric Interrupts and IRQ_F2P[15:0]. Click OK.

10. Make a connection from ip2intc_irpt to IRQ_FP2[0:0] by holding down the left mouse button on one of the pins and draw a line to the other pin while still holding the button. The pointer should turn into a pen when the connection is made. A green check mark will show up at pins that are possible end points. Here is the connection made.

12. We also need to create external connections for the second GPIO channel that we can hook up to the rest of the logic in the PL. Right-click on the gpio2 interface symbol on the AXI GPIO block and select Make External. Select the gpio2 port symbol and rename the external interface from gpio2 to Pushbuttons in the External Interface Properties.

12. Here is the result.

13. Make the FCLK_CLK1 an external signal.

14. Improve the appearance by selecting Regenerate Layout and verify the connectivity by clicking Validate Design.

15. Save the Block Design by clicking Ctrl-S.

We have now added both an interrupt signal and an external port to the AXI GPIO. The interrupt signal has already been connected internally in the block design but the second gpio port is external to the block design. In the next part we will connect it to a HDL module.

Adding push buttons

We will modify and add HDL sources to connect the push buttons. When connecting pushbuttons, it is wise to debounce the signals. We will import a HDL module describing a debounce circuit circuit and connect the push buttons through it.

1. Since we have modified the block design we need to regenerate the HDL files that are required for implementation, simulation and synthesis. Expand Design Sources and system_wrapper (system_wrapper.v) in the Sources pane, right-click system( and select Generate Output products. Click Generate.

2. Next we need to import the new HDL module that will debounce the pushbutton signals. We have already added a 250KHz clock from the PS that will be used to clock the debounce circuit and capture the pushbutton signals. The Verilog file debouncer.v can be downloaded from here (new version 2014-05-28).

3. To add the debouncer.v file to our design in the Flow Navigator pane select Project Manager->Add Sources.

4. Find and add the debouncer.v file.

5. Make sure that the box for Copy source into project is ticked and click Finish.

Modify the system_wrapper.v file

We need to modify the system_wrapper.v file to add our new debounce module. Double-click the system_wrapper.v file in the Design Source pane to open it in the Editor pane. We will make the following modifications:

  • Add an instantiation of the the debounce circuitry.
  • Add the FCLK_CLK1 signal
  • Connect clock, input and outputs to the rest of the system
  • Add a new input for the push buttons.
  • Connect debounce signals to the PROBE output
Save system_wrapper.v by typing ctrl-S. Here is the modified system_wrapper.v file (new version 2014-05-28).

RTL analysis

To get a graphical representation of our RTL code select RTL Analysis and click "Open Elaborated Design".

Synthesize the design

We need to clarify which external package pins that should be used to connect the input signals from the pushbuttons on the PCB. In order for Vivado to pick up all the signal names the design needs to be synthesized before I/O planning layout can be performed.

1. In the Flow Navigator pane, select Synthesis->Run Synthesis. The synthesis will take a few minutes to complete. Ignore any warnings. If we get an error we have to go back and check our design. Hopefully we get this message.

2. Change to Open Synthezised Design and click OK.

Connect package pins and implement the design

There are five push buttons connected to the PL. They are found here.

They are connected to the following package pins on the Zynq device.

3. Change to I/O planning layout in the toolbar. Follow the instructions in part 18 and add the five push buttons. Here is the result.

Generate bitstream

After fixing the package pins we are ready to generate a new bitstream file.

4. In the Flow Navigator pane, select Program and Debug->Generate Bitstream. Click Yes to run implementation. This will take a few minutes to complete. Ignore and close any warnings. When completed this window will be displayed.

5. Click OK to Open the Implemented Design. Must be open for exporting the bitstream file.

Export to SDK

6. Select File->Export->Export Hardware for SDK

Make sure to include the bitstream and click OK.

This completes the hardware session. In the next part we will add interrupt handling to our C application.

Top   Previous   Next

Posted at 20:25 by svenand
Comments (5)  

Monday, May 05, 2014
Zynq design from scratch. Part 45.
Installing Vivado 2014.1 and SDK 2014.1

A few weeks ago Xilinx released Vivado 2014.1 and SDK 2014.1 Let's install them and find out if anything has changed and what is new. Here are the release notes.


We can find all the documentation on the Xilinx support page. The first thing to read is the "Vivado Design Suite User Guide Getting Started". For more information see the following documents:
  • Vivado Design User Guide: Design Flows Overview (UG892)
  • Vivado Design User Guide: Embedded Processor Hardware Design (UG898)
  • Vivado Design Suite Tutorial: Embedded Hardware Design (UG940)
  • Vivado Design Suite User Guide: Designing IP Subsystems Using IP Integrator (UG994)
  • Vivado Design Suite User Guide: Using the Vivado IDE (UG893)
  • Vivado Design Suite User Guide: Design Analysis and Closure Techniques (UG906)


We will follow the instructions in part7 and only document things that have changed.

1.Download Xilinx_Vivado_SDK_Lin_2014.1_0405_1.tar.gz

2. Unzip, unpack and start the installer.

3. Select Vivado WebPACK.

4. Add the Software Development Kit (SDK)

5. Start the installation.

6. Here is the final result.

Edit .bashrc

We will add the following lines to the .bashrc file:


For new purchases of the Vivado Licenses beginning in 2014.1, Vivado Licenses use an activation-based licensing scheme. Activation-based licensing offers robust security and return features, as well as pave the way for future licensing enhancements. For more information see the Vivado release notes (chapter 5).

Activation based licenses

Instead of requiring a file to be present to authorize a machine, Activation uses a trusted area on the client or server’s hard-drive to store the authorization credentials. This trusted storage area will automatically be installed and initialized beginning with the Vivado 2014.1 installer. When the Vivado tools look for a license feature, they are allowed to run if this trusted storage area contains the proper authorization.

License compatibility

Both certificate and activation-based licenses will be recognized by Vivado 2014.1 and later versions. If the license versions and dates are valid for the tool version being used, it does not matter whether the license is certificate or activation-based. For example, a certificate-based license issued during a previous Vivado release, which is still within the 1-year subscription period will authorize Vivado 2014.1 or later software. No conversion to a new license methodology will be necessary during the remainder of a previous subscription period.

Getting a new license file

Although we can continue using our old license file I see some advantages using an activation based license. Let's get one. Follow the instructions in part 7 to generate and download a new license file. This time we select an activation based license.

After generating the license file, it is downloaded to our $HOME/Download directory. The file is called Xilinx.lic and looks like this.

To me it looks like an XML file. Let's rename it to Xilinx.xml.

Using the Vivado License Manager

The Vivado License Manager is provided on computers with the Xilinx Vivado tools loaded. Use the command vlm to start the license manager.

-> vlm

1. Select Load Design.

3. Click "Activate License" and find the license file (Xliinx.xml).

4. Click Open to load the XML file to the trusted storage area.

Starting Vivado

-> vivado &

Here is the new start up window.

If you get a segmentation error when starting Vivado see part 8 for an explanation.

Starting SDK

-> xsdk &

We are ready to use the Vivado 2014.1 release. I have rerun lab1 and lab2 without any major problems. Almost everything worked as expected. Here are two small hiccups.

1. When starting Vivado the following errors were reported.

Easily fixed by changing the permissions for the 2014.1 directory.

-> sudo chmod 777 $HOME/.Xilinx/Vivado/2014.1

2. When running an application in SDK the following window was displayed.

Easily fixed by deleting the run configuration and creating a new one.

Top   Previous   Next

Posted at 17:54 by svenand
Comments (8)  

Sunday, May 04, 2014
Zynq design from scratch. Part 44.

Time for a new suprise. Once again a parcel arrived and when unpacking it I found this box.

When opening the box I found the MicroZed board.

MicroZed™ is a low-cost development board based on the Xilinx Zynq®-7000 All Programmable SoC. Its unique design allows it to be used as both a stand-alone evaluation board for basic SoC experimentation, or combined with a carrier card as an embeddable system-on-module (SOM). MicroZed contains two I/O headers that provide connection to two I/O banks on the programmable logic (PL) side of the Zynq®-7000 AP SoC device. In stand-alone mode, these 100 PL I/O are inactive. When plugged into a carrier card, the I/O are accessible in a manner defined by the carrier card design.

The carrier card


All the documentation can be found at the website.

MicroZed overview

Quick start

The MicroZed board comes with a pre-installed Linux image stored on the SPI flash. When we power up the board the Linux OS will boot automatically. Here is what we have to do.

The microSD card

The MicoZed board has a holder for a microSD card. When inserted during boot up the card will be mounted under /mnt. This means that we can install programs on the SD card from our Ubuntu host that can then be executed from the Linux installation running on the MicroZed.

Copy files to the microSD

We insert the microSD into the adapter and put it in our SD card reader/writer. For more information about connecting to the Ubuntu host see part 38.

We will add a bin directory and copy the PrimeNumber application to the microSD card.


1. Verify the MicroZed boot mode (JP3-JP1) jumpers are set to QSPI card mode.

2. Connect an ethernet cable between the board and the ethernet switch.

3. Connect an USB cable between the board and the host computer. This cable connects  both power (+5V) and the terminal.

Terminal setup

We will use GTKterm as console. Start GTKterm in the Ubuntu guest OS (see part 14).

-> gtkterm &

Select port and set baud rate to 115200.

If everything works as expected we will see the zynq prompt in the console window.

Ethernet connection

We will change the MicroZed IP address to the same subnet as the rest of our system.

Now we can ping the board from our host computer.

FPGA configuration

Compared to the ZedBoard, the MicroZed does not have built-in USB-JTAG circuitry. There is a standard Xilinx PC4 connector for use with an external cable. Digilent has two JTAG programming cables we can use (HS1 and HS2). I will use the HS1 JTAG programming cable.

Install Digilent Adept JTAG drivers

Xilinx uses software from Digilent to configure Xilinx logic devices, initialize scan chains, program FPGAs, CLPDs and PROM. We will go to the Digilent web page and download all software from there. I know that some of this software is hidden somewhere in the SDK installation (/opt/Xilinx/SDK/2013.4/data/xicom/cable_drivers) but I prefer to do this installation from scratch and try to understand what is going on. For more information see part 13.

Connect to MicroZed board

We connect the programming cable between the PC4 connector and the USB port on our host computer.

PC4 connector

Jumper settings

We set the jumpers to cascaded JTAG chain.

VirtualBox settings

To make sure the USB device is recognized by our Ubuntu guest we open the VirtualBox settings and add the new USB device (Digilent Adept USB Device [0700].

After rebooting Ubuntu we can use the following commands to see if we have a working connection with the MicroZed board.

JTAG interface

You may wonder why there are two devices found. Here is the explanation.
The Zynq-7000 family of AP SoC devices provides debug access via a standard JTAG (IEEE 1149.1) debug interface. Internally, the AP SoC device implements both an ARM debug access port (DAP) inside the Processing System (PS) as well as a standard JTAG test access port (TAP) controller inside the Programmable Logic (PL). The ARM DAP as part of ARM CoreSight debug architecture allows the user to leverage industry standard third-party debug tools. For more information see the Zynq-7000 Technical Reference Manual (chapter 27).


This is all I planned to say about MicroZed for the moment. Most of the work we have done using ZedBoard can easily be transferred to MicroZed. Good luck!

Want to know more

For all of you using the MicroZeb board I recommend Adam Taylor's blog:
"Bringing up the Avnet MicroZed with Vivado".

Top   Previous   Next

Posted at 16:50 by svenand
Comment (1)  

Saturday, May 03, 2014
Zynq design from scratch. Part 43.
Hardware debugging

When you read this blog it may look like everything just works, but that is not always the case. I have spent many hours trying to understand what is going on inside my ZedBoard. For software debugging we have full access to almost all internal registers and can read and write to memory and set breakpoints and single step our programs. When it comes to hardware debugging we are much more restricted. Let's start by studying the document: "Vivado Design Suite User Guide, Programming and debugging"


Debugging an FPGA design is a multistep, iterative process. Like most complex problems, it is best to break the FPGA design debugging process down into smaller parts by focusing on getting smaller sections of the design working one at a time rather than trying to get the whole design to work at once. Iterating through the design flow by adding one module at a time and getting it to function properly in the context of the whole design is one example of a proven design and debug methodology. We can use this design and debug methodology in any combination of the following design flow stages

  • RTL-level design simulation
  • Post-implemented design simulation
  • In-system debugging
  • Using external instruments

RTL design simulation

The design can be functionally debugged during the simulation verification process. Xilinx provides a full design simulation feature in the Vivado® IDE. The Vivado design simulator can be used to perform RTL simulation of our design. The benefits of debugging our design in an RTL simulation environment include full visibility of the entire design and ability to quickly iterate through the design/debug cycle. The limitations of debugging our design using RTL simulation includes the difficulty of simulating larger designs in a reasonable amount of time in addition to the difficulty of accurately simulating the actual system environment. For more information about using the Vivado simulator, refer to the Vivado Design Suite User Guide: Logic Simulation (UG937)

Post-implemented design simulation

The Vivado simulator can also be used to simulate the post-implemented design. One of the benefits of debugging the post-implemented design using the Vivado simulator includes having access to a timing-accurate model for the design. The limitations of performing post-implemented design simulation include those mentioned in the previous section: long run-times and system model accuracy.

In-system logic design debugging

The Vivado IDE also includes a logic analysis feature that enables us to perform in-system debugging of the post-implemented design an FPGA device. The benefits for debugging our design in-system include debugging our timing-accurate, post-implemented design in the actual system environment at system speeds. The limitations of in-system debugging includes somewhat lower visibility of debug signals compared to using simulation models and potentially longer design/implementation/debug iterations, depending on the size and complexity of the design.

ChipScope Pro and the Serial I/O toolkit

ChipScope™ Pro tool inserts logic analyzer, system analyzer, and virtual I/O low-profile software cores directly into our design, allowing us to view any internal signal or node, including embedded hard or soft processors. Signals are captured in the system at the speed of operation and brought out through the programming interface, freeing up pins for your design. Captured signals are then displayed and analyzed using the ChipScope Pro Analyzer tool.

The ChipScope Pro Serial I/O Toolkit provides a fast, easy, and interactive setup and debug of serial I/O channels in high-speed FPGA designs. The ChipScope Pro Serial I/O Toolkit allows us to take bit-error ratio (BER) measurements on multiple channels and adjust high-speed serial transceiver parameters in real-time while your serial I/O channels interact with the rest of the system. To use ChipScope Pro we need a seperate license not included in Vivado WebPACK.

Debugging using external instruments

The possibility to analyze what is going on inside our Zynq device is valuable, but when we want to measure what is happening on our ZedBoard, we have to rely on external instrumentation. Professional logic analyzers and oscilloscopes are expensive instruments (>$1000) and maybe not reachable for our hobby projects. Let's look for a cheaper solution. When looking around I found LogiScope from Oscium.

LogiScope logic analyzer

LogiScope transforms an iPhone, iPad or iPod into a 100MHz, 16 channel logic analyzer. Not only is it the most intuitive logic analyzer available, the triggering is so powerful we'll be able to count the hair on our bugs (not my words). Specifications:

  • 100 MHz, 16 channel logic analyzer
  • Two logic harnesses (each with 8 digital, 1 ground)
  • Protocol decoding: I2C, SPI, UART, Parallel
  • Record length : 1000
  • Maximum input bandwidth : 30MHz
  • Works with 2.0v, 2.5v, 3.3v & 5.0v systems
  • Portable: goes where you go
  • Input voltage -0.5v to +7v
  • Advanced triggering
  • Compatible with Lightning and 30 pin connectors (with adapter, works with iPhone 5 & iPad Mini
  • Price : $200 (€150)

Looks like a perfect instrument for our project. Let's order one. If you live in Europe you can order it from Lab eSHOP. In the rest of the world it can be ordered from the Oscium webpage.


It took a few days and then this box appered on my desk.

Unpacking and connecting

Unpacking and connecting to the ZedBoard took only a few minutes. I have the first generation iPad which I used in my setup.

A closer look at the connection to the ZedBoard. As you can see I am using the Pmod connectors JA1 and JB1. The black wires are connected to ground.

Download and install

The iPad or iPhone app running the Logic Analyzer can be downloaded for free from the Apple AppStore (search for LogiScope).

Adding probes to the design

We have to modify our design to bring out the signals we would like to look at to the Pmod connectors. Here is a description on how to do that. 

1.Find out which Zynq pins are connected to the Pmod connectors. The information can be found in the ZedBoard Hardware Guide.

2. Start Vivado and open the LED_Controller project.

3. Open the system_wrapper.v file and add the PROBE output. We will connect the LEDS PWM signals and the FCLK_CLK0. The unused outputs will be forced to both 0 and 1.


   output [15:0] PROBES;

// Probes used for logic analyzer
   assign PROBES = {7'b1010101,FCLK_CLK0,LEDS};

4. Save the system_wrapper.v file (ctrl-S).

5. Follow the instructions in part 18 to synthesize the design and add new pin locations.

6. Connect the PROBE outputs to the Pmod connectors.

7. Run implementation and generate the new bitstream file.

8. Export design to SDK. Make sure the new bitstream file is exported.

9. Start SDK and connect to the new hardware design.

10. Follow the instructions in part 19 to load the new hardware design and to run the LED_Dimmer program.

11. Turn on the iPad and start the LogiScope app. Select different settings from the terminal and watch the pulse width changing.

To get started

Learn how to get started from this blog at EEweb.

Triggering demos

Oscium has put up three videos on their webpage illustrating the power of the advanced triggering system.


The LogiScope logic analyzer is very easy to setup and to use. It has its limitation when it comes to analyzing high-speed clocks > 30MHz but we can always add some prescaler logic in the programmable logic part before bringing out the signals. I think this instrument can be very useful in our Zynq design project.

Top   Previous   Next

Posted at 21:02 by svenand
Make a comment  

Wednesday, April 30, 2014
Zynq design from scratch. Part 42.
Writing userspace IO device driver

UIO Drivers

Linux provides a standard called UIO (User I/O) framework for developing user-space-based device drivers. The UIO framework defines a small kernel-space component that performs two key tasks:

a.    Indicate device memory regions to user space.
b.    Register for device interrupts and provide interrupt indication to user space.

The kernel-space UIO component then exposes the device via a set of sysfs entries like /dev/uioXX. The user-space component searches for these entries, reads the device address ranges and maps them to user space memory.

The user-space component can perform all device-management tasks including I/O from the device. For interrupts however, it needs to perform a blocking read() on the device entry, which results in the kernel component putting the user-space application to sleep and waking it up once an interrupt is received.


If we use UIO for our card's driver, here's what we get:
  • only one small kernel module to write and maintain.
  • develop the main part of our driver in user space, with all the tools and libraries we're used to.
  • bugs in our driver won't crash the kernel.
  • updates of our driver can take place without recompiling the kernel.

Creating an UIO driver

In this section we will create an UIO driver using the UIO framework.  We will modify the program used in part41 and call it: LED_DimmerUIO.c.

1. We start by generating a new application called LED_DimmerUIO.

-> cd ..../PetaZed
-> petaliniux-create -t apps --name LED_DimmerUIO

2. Replace the template file LED_DimmerUIO.c with the file downloaded from this page.

Configure user application

3. Run petalinux-config -c rootfs and select the LED_DimmerUIO application.

-> petalinux-config -c rootfs

4. Save and exit

Configure the Linux kernel

5. Run petalinux-config -c to start kernel configuration

-> petalinux-config -c kernel

6. Scroll down and select Device Drivers.

7. Scroll down and select Userspace I/O drivers.

8. Mark with an "M" for module.

9. Save and exit.

Edit the system.dts file

We find the device tree definition file system.dts file here:

10. Open the system.dts file in an editor and change the line after #gpio-cells = <2> to compatible = "generic-uio";

Rebuild the Linux kernel

The first time after we added the UIO module and edited the DTS file we have to rebuild the kernel from scratch.

-> petalinux-build -x mrproper
-> petalinux-build

When the build has finished we copy the image.ub file to the SD card and we are ready to boot PetaLinux.

Boot PetaLinux

Insert the SD card, power on the board and connect a terminal. Here are the commands executed to load the UIO module:

Top   Previous   Next

Posted at 13:31 by svenand
Make a comment  

Saturday, April 26, 2014
Zynq design from scratch. Part 41.
Petalinux board bringup

One of the great strengths of an FPGA platform is the ability to completely customise our processor system architecture, either for an off-the-shelf evaluation board, or for our own custom designs. PetaLinux SDK was created to embrace that flexibility, and includes a set of tools specifically designed to make it as easy as possible to boot a Zynq or MicroBlaze Linux platform on a new board or CPU subsystem design.

Let's build our own PetaLinux for the LED Controller design in lab2.


We will follow the procedure described in the document PetaLinux SDK Board Bringup Guide.

Hardware platform

Hardware platforms can be created from a number of sources, such as an existing Vivado/XPS project. Regardless of how the hardware system is created and configured, there is a small number of hardware IP and software platform configuration changes required to make the hardware system Linux ready. These are described below.

  1. One Triple Timer Counter
  2. External memory controller with at least 32MB of memory
  3. UART for serial console
  4. Non-volatile memory (QSPI Flash, SD/MMC)
  5. Ethernet

Our design fulfills all these requirements.

Configure software settings and bootloader

After designing the hardware system, it is necessary to configure a PetaLinux BSP for the platform. This is required to automate the PetaLinux board bringup. We will use Xilinx SDK to setup the PetaLinux Board Support Package (BSP) and the First Stage BootLoader (FSBL).

Launching Xilinx SDK

->xsdk &

Add PetaLinux SDK repositories

To add PetaLinux SDK repository select from the top menu:

Xilinx Tools->Repositories

Click the New button and browse to the directory: /opt/PetaLinux/petalinux-v2013.10-final/components/edk_user_repository

Click OK to finish adding local repositories.

Create Zynq FSBL

See part 40 for a decription on how to add the first stage bootloader program.

Create PetaLinux BSP

We will create the BSP and configure it for U-boot and the Linux operating system.

1. Select from the top menu:

File->New->Project and select the Board Support Package wizard.

Click Next. The New Board Support Package Project window appears. Select PetaLinux as BSP OS.

Click Finish and wait for settings window to open.

Select Petalinux from the left sidebar.

Change the settings for:

  • stdout
  • stdin
  • main_memory
  • flash_memory
  • gpio
  • sdio
  • ethernet

Click OK to finish the PetaLinux BSP setup. The directories fsbl_0 and petalinux_bsp_0 have been added to our workspace.

Create a new PetaLinux platform

The next step is to create a new PetaLinux SDK software platform, ready for building a Linux system customized to our new hardware platform. The petalinux-create command is used to achieve this:

-> petalinux-create --type project --template <CPU_TYPE> --name <PROJECT_NAME>

CPU_TYPE      (zynq or microblaze)
PROJECT_NAME  (the name of the platform we are building)

This command will create a new PetaLinux project folder from a default template. Later steps customize these settings to match the hardware project created previously. Here is the result:

Configure the software platform

The final step is to customize the software platform template to precisely match our unique hardware system. This is done by copying and merging the platform configuration files generated during the hardware build phase, into the newly created software platform, as described below:

1. Navigate to appropriate directory before using the petalinux-config command. In general the XSDK BSP directory is for example: "<XSDK workspace directory>/petalinux_bsp_0/"

2. Use the petalinux-config command to import the hardware configuration.

-> cd .../workspace/petalinux_bsp_0
-> petalinux-config --get-hw-description -p <plnx-proj-root>

The -p option points to the PetaLinux project that will be updated to match the hardware platform configurations. This step may take a few minutes to complete. This is because it merges the existing platform configuration to Kernel configuration and enables the appropriated drivers in the Kernel configuration.

This tool generates the hardware configuration files, if required and copies the configuration files to the correct location in our project directory. The configuration files are stored in the directory: PetaZed/subsystem/linux/hw-description

3. Change into the PetaLinux project root:

-> cd .../PetaZed

4. Launch the top level system settings configuration menu. Change the boot device to SD card.

-> petalinux-config

5. Launch the Linux kernel configuration menu. No changes made now.

-> petalinux-config -c kernel

5. Launch the rootfs configuration menu. No changes made now.

-> petalinux-config -c rootfs

Build the system image

Run petalinux-build to build the system image:

The compilation log: build.log is stored in the build directory. The result of the build can be found in the images/linux directory.

Generate BOOT.BIN image

Use this command to generate the boot image:

-> petalinux-package --boot --fsbl <FSBL image> --fpga <FPGA bitstream> --uboot

The FSBL image file and the bitstream files can be found here:

Boot system image on the ZedBoard

We will boot from SD card.

1. Copy the files BOOT.BIN and image.ub to the SD card
2. Insert the SD card into the ZedBoard.
3. Set the jumpers to SD boot
4. Connect a terminal
5. Power up the board.

Our Linux image will boot in a few seconds.

Running the LED_Dimmer application

We can not run the bare-metal LED_Dimmer application program inside the PetaLinux OS. We have to rewrite it to use direct memory accesses. Let's find out how to do it. We will start by studying the GPIO documentation. Here is the GPIO register map:

We find the GPIO base address in the PetaZed/hw-description/xparameters.h file

#define XILINX_GPIO_BASEADDR      0x41200000

Using the peek/poke program

Using the poke command we can write to any memory/register location in the Zynq address map. We can use this command to turn on/off the LEDs and set the dimming factor.

Writing Linux device drivers

I wrote a blog entry about this subject a few years ago. It is still valid. Let's study the poke.c source code,  found in the directory: /opt/Petalinux/petalinux-v2013.10-final/components/apps to get some ideas.

Userspace access via /dev/mem

"/dev/mem" is a virtual file representing the memory map of the whole system. To access the device from user space, we can open "/dev/mem", and then use mmap() to map the device to memory, and then we can access the device by using the pointer which points to the mapped memory. Here are some of the characteristics:

  • Userspace interface to system address space
  • Accessed via mmap() system call
  • Must be root or have appropriate permissions
  • Quite a blunt tool, must be used carefully
  • Can bypass protections provided by the MMU
  • Possible to corrupt kernel, device or other processes memory


  • Very simple, no kernel module or code
  • Good for quick prototyping / IP verification
  • peek/poke utilities
  • Portable (in a very basic sense)


  • No interrupt handling possible
  • No protection against simultaneous access
  • Need to know physical address of IP

OK for prototyping, not recommended for production

Rewriting LED_Dimmer.c

Here is a version of LED_Dimmer.c we can run in PetaLinux. To use it:
  1. Create a new application (LED_Dimmer) and add this program
  2. Run petalinux-config -c rootfs and add the LED_Dimmer application
  3. Build a new Linux kernel.
  4. Boot PetaLinux
  5. Execute LED_Dimmer

Top   Previous   Next

Posted at 16:59 by svenand
Comment (1)  

Friday, April 25, 2014
Zynq design from scratch. Part 40.
Booting from SD card or SPI flash

In a true, embedded application, we will not have a JTAG cable connected that can transfer our programs to the board. Our code must be able to do this before transferring control to an application. Let's return to lab2 and see how we can run our LED dimmer application directly from the SD card or from the on-board SPI flash.

But first we will learn a little bit more about the Zynq boot process. We can find a description in the documents "Zynq-7000 All Programmable Software Developers Guide (ug821)" and the "Technical Reference Manual (ug585)".

The processor system boot

The processor system boot is a two-stage process:
  • An internal BootROM stores the stage-0 boot code, which configures one of the ARM processors and the necessary peripherals to start fetching the First Stage Bootloader (FSBL) boot code from one of the boot devices. The programmable logic (PL) is not configured by the BootROM. The BootROM is not writable.
  • The FSBL boot code is typically stored in one of the flash memories, or can be downloaded through JTAG. BootROM code copies the FSBL boot code from the chosen flash memory to On-Chip Memory (OCM). The size of the FSBL loaded into OCM is limited to 192 kilobyte. The full 256 kilobyte is available after the FSBL begins executing.

The FSBL boot code is completely under user control and is referred to as user boot code. This provides us with the flexibility to implement whatever boot code is required for our system.

First stage bootloader

The First Stage Bootloader (FSBL) starts after the boot. The BootROM loads FSBL into the OCM. The FSBL is responsible for:
  • Initializing with the PS configuration data that Xilinx hardware configuration tools provide
  • Programming the PL using a bitstream (if provided)
  • Loading second stage bootloader or bare-metal application code into DDR memory
  • Handoff to the second stage bootloader or bare-metal application
Flow diagram

Create the first stage bootloader

The first step is to create the FSBL application. This is a C program that embeds all the Zynq internal register settings that were established during the Vivado Block Design. We start Xlinx SDK and make sure we have exported the lab2 design from Vivado and that we setup the Board Support Package (BSP).

->xsdk &

The SDK program will open with the setup we used in our lab2 experiment. Before creating the FSBL file we have to add a library file used by the FSBL c-program. Select from the top menu:

Xilinx Tools-> Board Support Package Settings

Click OK and wait for the settings window to open.

We will add the xilffs library. We are now ready to generate the FSBL program.

1. Select New->Application Project

2. Enter a name (fsbl_0) and select existing BSP (standalone_bsp_0).

3. Click Next.

4. Select Zynq FSBL and click Finish. When the generation has finished there is a new entry in the Project Explorer namned fsbl_0.

5. Right-click the fsbl_0 entry and select: Build Configurations->Set Active->Release. The release configuration will have less overhead.

6. Build the release configuration by right-clicking and select Build Project.

7. Here is the result:


Generate the boot image

The next step is to create a non-volatile boot image for ZedBoard. The ZedBoard has two non-volatile bootable sources, QSPI flash and SD Card.

1. In the Project Explorer select LED_Dimmer

2. From the top menu select: Xilinx Tools->Create Zynq Boot Image

3. The tool will automatically pick up the files needed to build the boot image.

  • fsbl_0.elf
  • system_wrapper.bit
  • LED_Dimmer.elf
4. To generate a boot image for the microSD card we will rename the Output path file name (output.bin) to boot.bin and click the Create Image button.

5. To generate a boot image for the SPI flash we will change the Output path file name to LED_Dimmer.mcs and rerun the Create Boot Image generation. It is the file type (bin or mcs) that defines what type of boot image that will be generated.

6. Here is the result.

Boot from the SD card

Copy the boot.bin file to the SD card and insert the card in to the ZedBoard. Set the jumpers to boot from SD card and power up the board. Connect a terminal and control the LEDs by entering different values.

Boot from SPI flash

First we will program the SPI flash using the LED_Dimmer.mcs file we just have generated. To program the SPI flash.

1. Connect the JTAG cable (see part 13).

2. Set the jumpers to JTAG mode.

3. In SDK select Xilinx Tools->Program Flash.

4. Specify the image file LED_Dimmer.mcs and click Program.

5. The programming will take a few minutes.

6. Power off the board and set the jumpers to SPI flash boot.

7. Power on the board. Connect a terminal and we are up and running in milliseconds.

Top   Previous   Next

Posted at 11:22 by svenand
Comment (1)  

Thursday, April 17, 2014
Zynq design from scratch. Part 39.
Running Android on ZedBoard

I was very impressed when I saw this video demonstrating how you can run Android on the Mars ZX3 Zynq module from Enclustra. If they can do it we can do it. Let's give it a try.

Work in progress

I am still working on this page. Changes will be done to fix mistakes I have made and to add new stuff.

A word of warning

Before you start implementing Android, study the last part of this blog entry. I am still waiting for a solution to the boot up problem. If you want to try something that works you should install Android 2.3 instead.

Android OS

Android is an operating system based on the Linux kernel and designed primarily for touchscreen mobile devices such as smartphones and tablet computers. Initially developed by Android, Inc., which Google backed financially and later bought in 2005.

Android's architecture diagram

Touchscreen display

To make full use of the Android operating system we have to add a touchscreen display to our ZedBoard. We have to figure out what is the best and cheapest solution but until then I will will continue the experiment without the display.


Before starting this experiment we have make sure we have at least 50GB free disk space. The Android installation itself, takes up almost 20GB and to build Android we need at least 30GB free disk space. If you have 50GB disk space available, congratulations. I don't have. I see two possibilities:

  1. Start all over with a new StorEdge card.
  2. Use shared folders and put the Andorid source code on the host computer.

My solution

I have decided to buy one more StorEdge 128GB card and setup a new VirtualBox virtual machine, making the virtual disk 120GB and then install Ubuntu 14.04 64bit (I couldn't find Ubuntu 13.10 any longer).

Recommended RAM size

In the documentation it says that 16GB RAM is recommended to compile and build Android 4.1. I have 4GB in my virtual machine. Will that last? No idea.

Read before we start

Here is the link to the official Android build documentation.

Install Ubuntu software packages

To be able to compile and build the Android system we have to install a number of software packages. We will follow this guide from xdadevelopers. Here is a command we can use to install everything in one run:

sudo apt-get install git-core lzop ccache gnupg flex bison gperf build-essential zip curl zlib1g-dev zlib1g-dev:i386 libc6-dev lib32ncurses5 lib32z1 lib32bz2-1.0 lib32ncurses5-dev x11proto-core-dev libx11-dev:i386 libreadline6-dev:i386 lib32z-dev libgl1-mesa-glx:i386 libgl1-mesa-dev g++-multilib mingw32 tofrodos python-markdown libxml2-utils xsltproc readline-common libreadline6-dev libreadline6 lib32readline-gplv2-dev libncurses5-dev lib32readline5 lib32readline6 libreadline-dev libreadline6-dev:i386 libreadline6:i386 bzip2 libbz2-dev libbz2-1.0 libghc-bzlib-dev lib32bz2-dev libsdl1.2-dev libesd0-dev squashfs-tools pngcrush schedtool libwxgtk2.8-dev python gcc g++ cpp gcc-4.8 g++-4.8 libswitch-perl

When that is done installing, run the following command in a terminal window:

-> sudo ln -s /usr/lib/i386-linux-gnu/mesa/ /usr/lib/i386-linux-gnu/

Java programming language

Most of the Android application source code is written using the Java programming language. Java is a computer programming language that is concurrent, class-based, object-oriented, and specifically designed to have as few implementation dependencies as possible. It is intended to let application developers "write once, run anywhere" (WORA), meaning that code that runs on one platform does not need to be recompiled to run on another.

Java was originally developed by James Gosling at Sun Microsystems (which has since merged into Oracle Corporation) and released in 1995 as a core component of Sun Microsystems' Java platform.

Java SE platform

One characteristic of Java is portability, which means that computer programs written in the Java language must run similarly on any hardware/operating-system platform. This is achieved by compiling the Java language code to an intermediate representation called Java bytecode, instead of directly to platform-specific machine code. Java bytecode instructions are analogous to machine code, but they are intended to be interpreted by a virtual machine (VM) written specifically for the host hardware.

Oracle Corporation is the current owner of the official implementation of the Java Standard Edition platform. This implementation is based on the original implementation of Java by Sun. The Oracle implementation is available for Mac OS X, Windows and Linux. Because Java lacks any formal standardization, the Oracle implementation is the de facto standard.

The Oracle implementation is packaged into two different distributions: The Java Runtime Environment (JRE) which contains the parts of the Java SE platform required to run Java programs and is intended for end-users, and the Java Development Kit (JDK), which is intended for software developers and includes development tools such as the Java compiler, Javadoc, Jar, and a debugger.

Java development kit

The Java Development Kit (JDK) is an implementation of either one of the Java SE, Java EE or Java ME platforms released by Oracle Corporation in the form of a binary product aimed at Java developers on Solaris, Linux, Mac OS X or Windows. In 2006, Sun announced that it would be released under the GNU General Public License (GPL), thus making it free software. This happened in large part in 2007, when Sun contributed the source code to the OpenJDK.

Download and install JDK

First, let's set up the correct JDK (Oracle's JDK). Many of us probably have some kind of wrong Java installed unless we're starting with a fresh Ubuntu base. Let's get rid of that. Copy and paste this into a Terminal window:

-> sudo apt-get purge openjdk-* icedtea-* icedtea6-*

Follow the instructions to remove OpenJDK. Now copy and paste the following into the Terminal:

-> sudo add-apt-repository ppa:webupd8team/java

This will add the correct PPA to our system for updated builds of Java 6 JDK that are compatible with Ubuntu 13.10. No more unrecognized Java version errors! And it will update automatically with the rest of our system. Next, we actually need to install the package. More copy-paste:

-> sudo apt-get update && sudo apt-get install oracle-java6-installer

Follow the on-screen instructions. We have to Accept the Licensing Agreement to complete the install. Let's make sure the correct version of Java is activated, run the following Terminal command:

-> java -version

Download and install Sourcery CodeBench

To build the Linux kernel we need to install the ARM toolchain. The Android source code includes a script to build the Linux kernel. This script is setup to use the GNU/Linux version of Sourcery CodeBench.

Sourcery CodeBench is a complete development environment for embedded C/C++ development on ARM, Power, ColdFire, and other architectures. Sourcery CodeBench Lite Edition includes:

  • GNU C and C++ compilers
  • GNU assembler and linker
  • C and C++ runtime libraries
  • GNU debugger

This Getting Started Guide explains how to install and use Sourcery CodeBench ARM GNU/Linux Lite.

1. Before starting the installation we have to install the following package:

-> sudo apt-get install libxst6:i386

2. Download the installer from the Mentor Graphics web page.

3. Select ARM processors and download the ARM GNU/Linux release.

4. Make the arm-2013.11-24-arm-none-linux-gnueabi.bin file executable.

->  chmod 755 arm-2013.11-33-arm-none-linux-gnueabi.bin

5. Start the installer as root.

-> sudo ./arm-2013.11-33-arm-none-linux-gnueabi.bin

6. Install the toolchain here:

7. Add the bin path to the PATH variable in the .profile file:

8. Add the following line in your .bashrc file

export CROSS_COMPILE=arm-none-linux_gnueabi-

9. Logout and login to enable the settings for the Ubuntu system.

Download and install Android source code

The Android source tree is located in a Git repository. This document describes how to download the source tree for a specific Android code-line. We will download Android 4.1 (Jelly Bean) modified for Zynq and ZedBoard.

Installing Repo

Repo is a tool that makes it easier to work with Git in the context of Android.

1. Make sure we have a bin directory in our home directory and that it is included in our path variable.

2. Download the Repo tool and ensure that it is executable:

-> curl > ~/bin/repo
-> chmod a+x ~/bin/repo

Initializing Repo

1. We will start by adding a project directory for our Android installation:

-> mkdir Projects/Android
-> cd Projects/Android

2. Configure GIT and add our email address and name:

-> git config --global
-> git config --global "Sven Andersson"

3. Initialize Repo with the GIT repository we will use:

-> repo init -u git:// -b jb -m default.xml

Downloading the Android source tree

To pull down the Android source tree to our working directory from the repositories as specified in the default manifest, run:

-> repo sync

The Android source files will be located in our working directory under their project names. The initial sync operation will take an hour or more to complete. Here is the result, almost 20GB of data has been downloaded.

Fixing the code

According to this guide the downloaded Android source code has some errors that must be fixed. Let's do that.

Building the system

After downloading the Android source code and installing the JDK we are ready to build our Android system. Initialize the environment with the script. Note that replacing "source" with a single dot saves a few characters, and the short form is more commonly used in documentation.

-> cd Projects/Android
-> source build/

Choose a target

Choose which target to build with lunch command. The exact configuration can be passed as an argument.

-> lunch

If run with no arguments lunch will prompt us to choose a target from the menu.

Build the code

Build everything with make. GNU make can handle parallel tasks with a -jN argument, and it's common to use a number of tasks N that's between 1 and 2 times the number of hardware threads on the computer being used for the build. E.g. on a dual-E5520 machine (2 CPUs, 4 cores per CPU, 2 threads per core), the fastest builds are made with commands between make -j16 and make -j32. I don't know how many parallel tasks I can handle due to my 4GB of RAM available in my virtual machine. Let's try this:

-> make -j4

Build result

The build was successful and took around 5 hours on my MacBookAir with an Intel i7 dual core processor running at 1.7 GHz (turbo boost 3.3 GHz). Here are some examples of memory and CPU utilization during the build phase.

The build produced the following output stored in the directory out:

The total size

The Android installation + build takes up almost 40GB of disk space.

Running the emulator

The emulator is added to our path automatically by the build process. To run the emulator, type:

-> emulator

Now we are only missing a touchscreen display.

Build the Linux kernel

With the ARM toolchain installed and the Android build completed we are ready to build the Linux kernel. Here is the command we use:

-> make -j4 out/target/product/zedboard/zImage

The build takes about 15 minutes and the following files have been generated and added to the zedboard directory:

  • devicetree_ramdisk.dtb
  • ramdisk8M.image.gz
  • zImage

Preparing the SD card

We will boot Android from a SD card inserted in to the ZedBoard. The next step to figure out, is how to format the SD card and which files to add to the card. We will use a 16GB SanDisk SDHC card.

Format the SD card

We make one partition formatted in FAT32. See part 38 for more information on how to format an SD card. We can also use the command fdisk and follow this description. Don't forget to unmount the card before you start.

Copy the following files to the SD card. The BOOT.BIN file is taken from the original SD card.

The whole system size is approximately 1.5GB.

Boot the Android system

Time to boot our system. Follow the instruction in part 38 to setup the ZedBoard and the peripherals. Switch on the power and wait for the blue LED to light up. Don't stop the booting and wait for the Android logo to be displayed on the display connected to the HDMI port.

This is how far I came

It just hangs there. Nothing more is displayed and on the terminal the following text is passing by.

I am not the only one having this problem. Here is an explanation but where is the solution. Help needed.

Top   Previous   Next

Posted at 07:56 by svenand
Comments (8)  

Previous Page Next Page