ConnextX missing - should I flash? #8
Comments
Please try running the board directly. A ~240GB SSD is enough for a complete Ubuntu system with Vivado. The tools require access to the PCIe Bridges as well.
Unfortunately the easiest course of action is to update the ConnectX-5 firmware. In its current state the Innova2 Flex Application and Drivers will only communicate with Please save multiple copies of the current firmware and just in case test the network ports both before and after updating the firmware. The Lenovo Firmware Release for the Innova2( Also, if you have a Xilinx-compatible JTAG adapter, save the current FPGA contents by running the following command in Vivado's Tcl Console (Vivado Lab will work as well and 2023.1 currently works):
Please post the output of Thanks for posting @AGenchev,
Great goal. I would be very interested in your progress.
QDMA has features for multi-user cloud computing like virtual functions and is intended for low-latency small-packet performance. XDMA is much simpler and designed for high-throughput bulk data transfer. Check out innova2_xdma_demo and qdma_2021_1. |
|
Thank you so much for the invaluable information ! You really inspire me to go further. Finally I managed to obtain a Xilinx-DLC10 JTAG adapter, installed the recommended ubuntu, vitis+vivado & all the drivers. I'm not sure why I need all drivers if I'm not using the network card. sudo lspci -vvvnnxxx output (https://gist.github.com/AGenchev/f466b047b2a9cd13bc1114208c3828d2#file-gistfile1-txt) But, it failed me again: -E- Burning FS4 image failed: Cannot burn device data sections, Flash is write protected. |
|
My next update: "A new hope" now I have the preffered PSID (MT_0000000158). After cold reboot, mlx drivers are still unhappy as they were before (mlx5_core hangs 2 times x3 minutes). And in the end gives me: Could it be because the ports don't have SFP modules attached ? HW manager (JTAG) also errors: Ran mlxup, after the following I gave it perm. to update to 16.28.2006: (Thoughts for if/after I manage to solve the issues: |
Until the driver for Recovery Mode is loaded. I fixed the notes to make that more clear.
Yes I recently tried the procedure on a board with a PSID different than
This is very difficult to get right as
This one is new for me. It is my understanding that the W25Q128JVS gets loaded once at boot then resides in CX5 memory.
The ConnectX-5 works well enough to update its own firmware which is good news. What is the FBGA Code on your board's DDR4 ICs (D9TBK or D9WFR)?
What is your cooling solution? The ICs get very hot very quickly which leads to all sorts of problems. Post a picture of your cooling setup.
That would just make the board hotter. I run mine with no SFP modules or cables and it works fine.
Did your board not include a label with the MAC IDs? use the values you saw when you first ran
Yes. I suspect it will take a while to figure out why. What is the motherboard and CPU you are using? Does it support at least PCIe 3.0?
Yes, without the ConnectX-5 Ethernet Controllers there is no way for the drivers needed by What does
I have been working on a Tutorial for XDMA and XDMA Communication. PCIe P2PDMA is a complex project I have not yet been able to get working. |
|
I wrote in chronological order; "..Flash is write protected." IMHO was caused by write protection indeed, because if you saw my first flint query, it said at the bottom:
It seems you have nailed it:
I simply attached 2 main board north-bridge radiators which have springs, using elastic rubber band to hold them in place and to provide pressure on the springs. Against the smaller one I put a blower fan, which provides enough airflow.
The App could start, reads temperatures: With FPGA power increased to 10 (what was that - overclock?) it went to 80 C. All tests from innova2_flex_app fail (on my 4GB version). ...reboot.. Sending 8k data to M_AXI BRAM Controller Block succeeded first time, then fails, same for the read op. The read/write ops are very slow: |
That makes sense and thanks for pointing it out. I have not come across this.
Likely. My guess is that the FPGA runs some large DSP-heavy project.
That still seems safe.
Most fail for me on the 8GB ADLT variant as well. Luckily the tests have no relevance to the usefulness of the Innova-2.
Yes, 4GB is enough for everything you could try with these boards. The 8GB ADLT has both SFP GbE interfaces connected to the ConnectX-5 with the intent of using PCIe P2PDMA for communication between the FPGA and Ethernet.
From the 4GB ADAT Product Brief(manualzz.com) it appears to have at least one direct FPGA SFP interface. This is good news but it means that the 8GB ADLT firmware is not designed for it.
Issue #3 has more on the differences between the boards. Does your board have any markings that indicate it is the ADAT or ADIT?
Yes, the bitstream from test_adit_mt40a512m16 will test 512MB of the 4096MB. I tried a straightforward port of innova2_8gb_adlt_xdma_ddr4_demo to the MT40A512M16 (D9TBK) but it fails. I need to run more tests to figure out why.
Great news!
Yes, that is the correct sequence to program a Configuration Flash Memory (MT25QU512 RW193 ICs) bitstream using Using JTAG to load a bitstream into the FPGA's Temporary SRAM Configuration Memory is useful for development but not required.
It looks like your FPGA Flash Configuration Memory already includes the latest Flex Image. What version does
No, your board seems to work. I had clear damage to my board.
After
That is good news.
More good news, XDMA is working for at least some time. My first instinct is that something is overheating and therefore fails after some time. Do you have access to a Thermal Imaging Camera or Freeze Spray? The idea with Freeze Spray is that you spray the board and the hottest parts will return to normal fastest. I notice you have an Nvidia Tesla in your system. Are you able to remove it and try testing again?
The driver cannot access the AXI Bus. If removing the Tesla card makes no difference or you cannot, try updating Uninstall Do you have access to a large ultrasonic cleaner, a well-ventilated space such as a balcony or garage, and 99% Isopropyl Alcohol? It has been my experience that old server equipment that is flaky can sometimes become usable after about a half hour of ultrasonic cleaning in Isopropyl Alcohol. |
my goal is to use ConnectX-5 as .. just a PCI-E bridge to allow the FPGA on the PCI-E bus.
I flashed it and succeeded in running your test at the point "I ran a sanity check and tried sending 1GB of data. Only 512MB transfers as expected." The results are repeatable, xdma works. The speed is ~ 50 MB/s when 512 MB of 1 GB are transferred which seems not too high. I didn't verify if the contents written/read match. The JTAG doesn't run as expected. The last test I did was to enable JTAG on the application and then to run xsdb% targets It doesn't discover the FPGA. Can I reverse the connector |
Check that the data transfers correctly. More and more of the board is proving to work. Let's hope the success continues.
Please post the output of
Constraints for the 8GB ADLT are known. When I use the same constraints in innova2_8gb_adlt_xdma_ddr4_demo but change the Memory Part of the DDR4 IP Block to If you have the time, try recreating
Are you able to communicate with other Xilinx FPGAs using your debugger? I am on my second 14-Pin JTAG cable(1). With everything powered off, check with a multimeter in continuity mode that all the JTAG signals connect between the Innova2 and your JTAG adapter and are not shorted to GND. |
|
Thank you for the guidance! I will be able to do all this after 7th JAN 2024, because I am on vacation far from home till then w/o access to IT equipment. |
|
I got DDR4 working on my Innova2 4GB ADIT board. The My problem turned out to be a reset issue. When I tried porting the |
This is really good news ! Thank you so much ! I'll try it. I only found time to test the JTAG adapter vs another board (an old NETFPGA-1G-CML) and it worked. This means I have also kind of problematic JTAG port on the innova2.
This means that version 2023.2 also works and generates working images. I have installed 2021. Next evening I'll test whether the JTAG connector pins aren't grounded and whether your bitstream works (even if the JTAG isn't working). Why I'm trying to get JTAG working - to skip reboot when I want to set a new image to the FPGA. |
|
Now I'm trying to understand some basics: the block scheme of 4gb_adit_xdma_ddr4_demo looks rather complex, so it seems it does more than just being a XDMA to int.DDR4 memory controller. I see for the memory you set a different arbitration scheme (RD PRI REG) on the ADIT board while on ADLT. I have to read much more to get what is going on. I think to create a very basic block design using the DDR-4 guidelines from your demo, then to add to it more blocks trying to generate something into DDR4 and read it. Not sure if 2 devices try to access the DDR4 slave AXI at the same time what will happen... |
Yes, I added various blocks to test other parts of the board.
Yes, a much simpler design should work in Vivado 2021.2 to test DDR4:
I plan to eventually update my
The AXI Arbitration Scheme should not matter for such a simple design. For designs with multiple AXI blocks that use DDR4 Round-Robin guarantees low-latency access at the cost of throughput. To be honest I forgot to change it from the default Read Priority. Refer to the Memory IP Guide (pg150) Pg#143.
Yes, unfortunately not. You can use an evaluation license or the AWS Vivado 2023.2 AMI by the hour.
The Innova-2 uses 1.8V for JTAG. Does your adaptor support that? I recommend one of the Officially-Supported JTAG Adaptors or the Waveshare Platform Cable clone. I am aware of inexpensive FT2232H-based JTAG SMT boards causing problems.
Working JTAG will also allow you to use XSCT/XSDB to debug soft-core processors like the RISC-V or MicroBlaze. |
|
I report JTAG success ! The JTAG adapters I tried are on the picture. First one I bought for US$17 on Aliexpress - it is a Chinese clone. The second one costs here $300 and I don't own it. Counterintuitive, the cloning performs slightly faster. You were right - my JTAG was not broken. I don't know now the reason it didn't work - very likely my misunderstanding the usage of innova2_flex_app: |
Excellent news!
Thanks for pointing that out. I have tried to improve my notes.
When you run
Good to know. I have added a comment to the innova2_4gb_adit_xdma_ddr4_demo project.
Yes, please do. Create new issues if you encounter any problems. |
Of course, but I had only 1 GB, because even though I use a PMR server HDD w 256MB cache, 32G swap would be awfully slow. And top now shows: (Vivado is running at background) |
Hello, someone handed me a lenovo-made innova2. It is missing the connect-x device.
Should I try unconventional methods on it to flash another image than the supplied by lenovo so development could be possible ?
I think there is a problem with the card, but I can't diagnose it for now.
Note: To install all required software versions, I set up a virtual machine where I added the 2 mlx PCI-E devices using pci-e passthrough. There is no third device neither on the host nor in the guest system.
my final goal is to setup a device on the fpga (w bus-master DMA) which will generate test data and stream it to the host memory. I'm still trying to learn what is QDMA and how it's different from XDMA, but for now I don't have what to test on :-)
The text was updated successfully, but these errors were encountered: