Skip to content

Run your first SDAccel program on AWS F1

Jump to bottom
Thomas Bollaert edited this page Sep 13, 2017 · 16 revisions

Overview

This tutorial explains the procedure to package an RTL design as an SDAccel kernel and then use this RTL kernel to accelerate a host application. The tutorial uses the vadd_kernel example from the SDAccel Github examples repository.

The following steps will be covered:

  1. Writing an RTL design adhering to the SDAccel kernel interface requirements
  2. Packaging the RTL design as an SDAccel kernel (XO file)
  3. Compiling the host application and the FPGA binary containing the RTL kernel
  4. Creating the Amazon FPGA Image
  5. Executing the host application with the Amazon FPGA image

Note

This tutorial doesn't presently use the SDAccel RTL Kernel Wizard. The SDAccel RTL Kernel Wizard is a new feature which assists users through the process of packaging RTL designs as SDAccel kernels. The RTL Kernel Wizard generates the required XML file, an example project design and a set of scripts to build that example design into an XO file. For more details on how to use the RTL Kernel Wizard, you can watch this online video

Preparing to run the tutorial

  • Execute the following commands to clone the Github repository and configure the SDAccel environment:
    $ git clone https://github.com/aws/aws-fpga.git
    $ cd aws-fpga                                      
    $ source sdaccel_setup.sh
  • Go to the testcase directory
   $ cd SDAccel/examples/xilinx/getting_started/rtl_kernel/rtl_vadd

The SDAccel Github examples use common header files and those needs to be copied in the local project source folder to make it easier to use.

  • Type the command make local-files to copy all necessary files in the local directory.
  $ make local-files

1. Writing an RTL design adhering to the SDAccel kernel interface requirements

To be used as an SDAccel kernel, an RTL design must comply with the following signals and interface requirements:

  • Clock.
  • Active Low reset.
  • 1 or more AXI4 memory mapped (MM) master interfaces for global memory. All AXI MM master interfaces must have 64-bit addresses.
    • You are responsible for partitioning global memory spaces. Each partition in the global memory becomes a kernel argument. The memory offset for each partition must be set by a control register programmable via the AXI4 MM Slave Lite interface.
  • One and only one AXI4 MM slave lite I/F for control interface. The AXI Lite interface name must be S_AXI_CONTROL.
    • Offset 0 of the AXI4 MM slave lite must have the following signals:
      • Bit 0: start signal - The kernel starts processing data when this bit is set.
      • Bit 1: done signal - The kernel asserts this signal when the processing is done.
      • Bit 2: idle signal - The kernel asserts this signal when it is not processing any data.
  • One or more AXI4-Stream interfaces for streaming data between kernels.

In this example, the RTL is already compliant and doesn't need to be modified.

The RTL code for this example is located in the ./src/hdl directory.

2. Packaging the RTL design as an SDAccel kernel (XO file)

A fully packaged RTL Kernel is delivered as an XO file which has a file extension of .xo. This file is a container encapsulating a Vivado IP object (including RTL source files) and a kernel description XML file. The XO file can be compiled into the platform and run in the SDAccel hardware or hardware emulation flows.

To package the kernel and create the XO file the following steps are required:

  • Writing a kernel description XML file
  • Packaging the RTL as a Vivado IP suitable for use in IP Integrator
  • Running the package_xo command to generate the XO file

Writing a kernel description XML file

A special XML file is needed to describe the interface properties of the RTL kernel. The format for the kernel XML file is described in the Create Kernel Description XML File section of the documentation.

This XML file can be created manually or with the RTL Kernel Wizard. In this example, the XML file is already provided (./src/kernel.xml).

  • Look at the content of the file to familiarize yourself with the information captured in the XML description.

Packaging the RTL as a Vivado IP suitable for use in IP Integrator

The example comes with the ./scripts/package_kernel.tcl script which takes the existing RTL design and packages it as a Vivado IP. The script places it in an IP directory called ./packaged_kernel_${suffix} where “suffix” is specified as an user argument.    

Running the package_xo command to generate the XO file

  • Run the following commands to package the RTL and create the XO file:
  $ vivado -mode tcl  

  # Set suffix for the directory for RTL-IP import   
  Vivado% set suffix rtl_ip    

  # Import the RTL to the “packaged_kernel_{$suffix}” IP directory   
  Vivado% source scripts/package_kernel.tcl   

  # Create the XO file
  Vivado% package_xo -xo_path ./src/rtl_vadd.xo \
                     -kernel_name krnl_vadd_rtl \
                     -ip_directory ./packaged_kernel_rtl_ip \
                     -kernel_xml ./src/kernel.xml
  # Exit Vivado
  Vivado% exit

The ./src/rtl_vadd.xo file gets generated. It contains all the necessary information SDAccel needs to use the kernel.

3. Compiling the host application and the FPGA binary containing the RTL kernel

This section covers the following steps:

  • Creating and configuring a new project
    • Starting the SDAccel GUI
    • Creating a workspace
    • Setting the platform
    • Creating a new empty project
    • Importing the application host code and kernel XO file
    • Specifying the binary container for the kernel executable
  • Verifying the application using the hardware emulation flow
  • Compiling the host application and the FPGA binary for hardware execution

The host application code for this example is in the ./src/host.cpp file.

In the SDAccel flow the host code uses OpenCL APIs to interacts with the FPGA.

Creating and configuring a new project

Starting the SDAccel GUI

  • Open the SDx GUI by running the following command:
  $ sdx

Creating a workspace

  • In the Workspace Launcher window, add a workspace inside the current directory named Test_dir as shown below. A new directory Test_dir will be created and used to store all the logfiles from our runs.

Setting the platform

  • In the Welcome window, click Add custom platform to set the path to AWS F1 platform.

  • Click on the plus sign as shown below.

  • Then browse to the /SDAccel/aws_platform/xilinx_aws-vu9p-f1_4ddr-xpr-2pr_4_0/ directory, select platform.

  • Click Apply and OK. This completes the platform setup process.

Creating a new empty project

  • In the Welcome window, click Create SDX Project and set the project name to TEST_RTL_KERNEL.
  • Move through the next three screens (keeping the default selections) by clicking Next -> Next -> Next.
  • Finally select an Empty Application in the Available Templates section, and then click Finish.

Importing the application host code and kernel XO file.

On the left side of the SDAccel GUI you will see the Project Explorer pane.

  • Right Click on project.sdx and then select Import.

  • Select General -> Filesystem and then click on Next.
  • Browse to the source file directory of the current example, rtl_vadd/src
  • Select the files host.cpp , xcl.c, xcl.h and rtl_vadd.xo as shown below.

Specifying the binary container for the kernel executable

In the center of the SDAccel GUI you will see the SDx Project Settings.

  • Click Add Binary Container the icon as shown below

  • In the top menu, select Run->Run Configuration
  • Select the Arguments tab and check the Automatically add binary container(s) to arguments box

The project creation and setup is now complete.

Verifying the application using the hardware emulation flow

SDAccel provides three different build configurations: Emulation-CPU, Emulation-HW and System.

In the Emulation-CPU mode, the host application executes with a C/C++ or OpenCL model of the kernel(s). The main goal of this mode is to ensure functional correctness of your application. Note: this mode is not presently supported for RTL kernels.

In the Emulation-HW mode, the host application executes with a RTL model of the kernel(s). This mode enables the programmer to check the correctness of the logic generated for the custom compute units and gives performance estimates.

In the System mode, the host application executes with the actual FPGA.

  • To run hardware emulation, go to SDx Project Settings and make sure that Active build configuration is set to Emulation-HW.

  • Click the Build icon to start the emulation build process.

  • After the emulation build process completes, run Hardware Emulation by clicking the Run Icon.

After completion of Hardware Emulation run, you can find and inspect various reports in the Reports tab, such as the System Estimate, Profile Summary, and Application Timeline.

Compiling the host application and the FPGA binary for hardware execution

  • To run hardware execution, go to SDx Project Settings and set Active build configuration to System.
  • Click the Build icon to initiate the hardware build process.

It generally takes few hours to complete the hardware build.

At the end of this process, the host executable (TEST_RTL_KERNEL.exe) and FPGA binary (binary_container_1.xclbin ) are generated in the System directory.

  • Exit the SDAccel GUI.

4. Creating the Amazon FPGA Image

In order to execute the application on F1, an Amazon FPGA Image (AFI) must first be created from the FPGA binary (*.xclbin). This step cannot be presently performed through the SDAccel GUI. The AFI is created using the AWS create_sdaccel_afi.sh command line script.

  • Using your S3 bucket, S3 dcp folder and S3 log folder information, execute the following command:
  $ SDACCEL_DIR/tools/create_sdaccel_afi.sh \
         -xclbin=binary_container.xclbin \
         -o=binary_container.awsxclbin \
         -s3_bucket=<bucket-name> \
         -s3_dcp_key=<dcp-folder-name> \
         -s3_log=<logs-folder-name>

The above step generates an *.awsxclbin file and an *_afi_id.txt file containing the ID of your AFI. The AFI ID can be used to check the status of the AFI generation process.

  • Rename the .awsxclbin to original .xclbin file.
  $ mv binary_container.awsxclbin binary_container.xclbin
  • Note your AFI ID
  $ cat <timestamp>_afi_id.txt
  • Check the status of the AFI generation process
  $ aws ec2 describe-fpga-images --fpga-image-ids <AFI ID>

The command will return Available when the AFI is created, registered and ready to be used. The command will return Pending otherwise.

    "State: { 
        "Code" : Available" 
    }

5. Executing the host application with the Amazon FPGA image

Once the AFI is Available, you can execute the application on the F1 instance.

  $ sudo sh
  # source /opt/Xilinx/SDx/2017.1.rte/setup.sh   
  # ./TEST_RTL_KERNEL.exe 
  Device/Slot[0] (/dev/xdma0, 0:0:1d.0)
  xclProbe found 1 FPGA slots with XDMA driver running
  INFO: Importing ./binary_container.xclbin
  INFO: Loaded file
  INFO: Created Binary
  INFO: Built Program
  All Device results match CPU results! 
  TEST PASSED.

Congratulations, you have completed your first SDAccel program on F1 using RTL kernels.

SDAccel Examples Wiki

Clone this wiki locally