Urukul v1.0 detailed testing

[marmeladapk]

From @hartytp on 2017-11-07 20:29

• Channel-to-channel cross-talk: adjacent channels, with agressor, victim switches opened and switches closed and with attenuators at max/min
• Output phase noise/amplitude noise (0.1 Hz - 10 MHz)
• OIP3, OIP2 vs frequency for min attenuator setting and medium/large amplitude (ASF)
• RF switch settling behavior for on edge at max power (worst case)
• Attenuator glitches
• Nyquist attenuation, clock leakage
• RF switch settling behavior for off edge at max power (worst case)
• RF switch phase chirp
• Temperature coefficient of power and phase
• Are we happy running the IC11 and running the LVDS interface directly from the 3V6 rail? (Try this and check performance)
• Check performance of clock buffers when running in LVDS mode from the 1V8A rail
• Optimize PLL loop filters (@WeiDaZhang can you do this for the AD9910 boards, please?)
• Board-to-board crosstalk (in proximity to Kasli, DIO pulsing 5 V into 50 R, Urukul)

Use some fixed reference frequency for all tests, such as 80MHz.

[marmeladapk]

From @hartytp on 2017-11-07 20:30

@gkasprow Can you ship us all but 1 of the AD9910 Urukul prototypes you've ordered as soon as basic functionality/"no smoke" testing is complete (i.e. don't complete the above detailed testing first!).

Thanks!

[marmeladapk]

From @hartytp on 2017-11-07 20:30

@gkasprow @jordens Feel free to add more tests to the above list.

[marmeladapk]

From @hartytp on 2017-11-08 03:32

@gkasprow When you ship the AD9910 DDSs that we paid for, please can you ship one to @jordens?

[marmeladapk]

From @jordens on 2017-11-08 08:47

Updated it a bit. Please ship right after smoke test:

• power supplies good
• clock fanout fans out
• dds pll locks
• dds outputs frequency
• attenuators can be set
• rf switches work

We'll be happy to help with characterization afterwards.

[marmeladapk]

From @hartytp on 2017-11-12 18:22

@gkasprow Is Urukul due to arrive this week? When it arrives, please can we prioritise no smoke/basic functionality testing (see reobert's list) above finishing Zotino -- I'd like to get Urukul prototypes shipped asap. No need to do the detailed testing for now.

Thanks!

[marmeladapk]

From @gkasprow on 2017-11-12 18:26

@hartytp OK, 2 Zotinos + BNC adapters were shipped on Friday. Once I receive DDS, I will test them ASAP.

[marmeladapk]

From @hartytp on 2017-11-12 18:27

Thanks!

[marmeladapk]

From @gkasprow on 2017-11-12 18:32

@hartytp There are no panels for DAC BNC adapters attached - I forgotten about them. Will ship them next time with some other stuff.

[marmeladapk]

From @hartytp on 2017-11-12 18:43

@gkasprow No problem. That can wait.

If there is any time before Urukul arrives, it'd be great to get a few of the analog test for Zotino done.

[marmeladapk]

From @gkasprow on 2017-11-12 19:04

@hartytp Sure, once I repair my bard, will continue with measurements.

[marmeladapk]

From @hartytp on 2017-11-12 19:23

Great, thanks @gkasprow!

[marmeladapk]

From @jordens on 2017-11-19 16:38

@hartytp could you remember to do a bit of triage on the issues you file (i.e. assignee, labels, milestone)?

[marmeladapk]

From @hartytp on 2017-11-19 18:48

sure

[marmeladapk]

From @hartytp on 2017-11-20 18:55

@gkasprow For the current source biasing, my inclination is to go for Daniel's suggestion of the LT3092. They're pretty cheap ($2 in quantity) and seem like a better/easier solution than building our own current source.

@jordens @gkasprow if you're oaky with this approach, then we should buy an eval board at some point and test this out with the v1.0 boards we have. I'm happy to do this testing if that helps -- I probably won't do this until after Christmas, but I don't think we're in a rush for it.

[marmeladapk]

From @jordens on 2017-12-11 18:24

@a-shafir and I will take care of the first round of noise, power, RF switch testing.

[marmeladapk]

From @hartytp on 2017-12-11 18:26

Cool.

Out of curiosity, do you have a proper phase noise/amplitude noise meter? Or, will you just stick this on a spectrum analyser?

[marmeladapk]

From @jordens on 2017-12-11 19:16

We have proper tools.

@gkasprow how are the AD9912 variant boards?

[marmeladapk]

From @gkasprow on 2017-12-11 21:11

@jordens I have them all in my lab but was busy with Sayma Ethernet debugging. You can follow my fight here

[marmeladapk]

From @gkasprow on 2017-12-11 21:12

So the next thing is 9912 testing.

[marmeladapk]

From @jordens on 2017-12-11 22:32

I am following that and involved.
Thanks for the AD9912 update. Once you have smoke-tested them like the AD9910 stuff, you can send two of them to me.

[marmeladapk]

From @a-shafir on 2017-12-11 22:45

@gkasprow what was actually flashed into the CPLD on the 9910 board we have in Berlin?

I see 2 options
https://github.com/m-labs/sinara/tree/master/ARTIQ_ALTIUM/EEMs/Urukul/CPLD_code
or
https://github.com/quartiq/urukul

I am testing h/w but can't generate any output on the IO_SESET (P129) and some pins around.

[marmeladapk]

From @a-shafir on 2017-12-11 23:02

@gkasprow how you tested 3U DDS 9910?
Have you got DDS ic working?
Do you have the code that sent the spi commands, what actually sent to DDS/atten etc?

[marmeladapk]

From @gkasprow on 2017-12-11 23:02

@a-shafir I burned Roberts Verilog code. The only change I did was assignment of test points. Now we have:

assign tp = (cfg_data_rst | ((~en_9910) & tstriple7_i));
assign tp_1 = dds_cs_n;
assign tp_2 = dds_sck;
assign tp_3 = dds_sdi;
assign tp_4 = dds_io_update;

[marmeladapk]

From @gkasprow on 2017-12-11 23:05

I connected 9910 devkit via DIO-Tester module to make sure that Robert's code works. In this way I could communicate with it using LVTTL signals

[marmeladapk]

From @gkasprow on 2017-12-11 23:06

I also modified the verilog code so all 4 channels were active simultaneously. In this way I tested if PLLs are locked. Here is the code:
Urukul_CPLD_all_ch.zip

[marmeladapk]

From @gkasprow on 2017-12-11 23:08

To do general purpose communication over SPI I use little Artix 7 devkit with USB and FORTH processor that translates UART commands to bit-bang protocol, i.e. SPI. It works really fast but it's written in FORTH, so probably not really useful for you :)

[marmeladapk]

From @a-shafir on 2017-12-11 23:13

@gkasprow i see io_reset commented in the code. probably some other changes.
Please also attach the pin assignment for the code.
So what was generating the DDS SPI command? 9910 devkit? I have no devkit.
Can you generate a dump (sequence) of the commands?
Also what about the reading from DDS when all in parallel?

[marmeladapk]

From @a-shafir on 2017-12-11 23:15

@gkasprow i see why in_sel not working to - it commented out
Good that i got this data from you )

[marmeladapk]

From @hartytp on 2018-02-01 20:05

We took some data today using the AD9910 Urukul with a Wenzel oscillator source. Will upload tomorrow.

[marmeladapk]

From @hartytp on 2018-02-02 16:13

Courtesy of @WeiDaZhang: Urukul AD9910 phase noise.
Urukul Phase Noise Measurement - Ver1.0 w- XTAL Disabled.pdf

[marmeladapk]

From @hartytp on 2018-02-02 16:15

Urukul noise performance basically matches the data sheet curve.

[marmeladapk]

From @hartytp on 2018-02-02 18:19

Urukul Amplitude Modulation Noise Measurement - Ver1.0 w- XTAL Disabled.pdf

Uruku AM noise. cf

[marmeladapk]

From @hartytp on 2018-02-02 18:24

Conclusions:

• the loop filter isn't quite optimal, which isn't a surprise since the VCO characteristics aren't particularly well specified (not even clear how well controlled they are). We could probably make the servo bump about 7dB smaller with optimisation, but probably not worth the effort.
• the LDO dropout voltage doesn't seem to have too much of an impact. There are a few spurs on the AM output, which may be SMPS leakage. Might be worth adding inductors between the SMPS and LDOs (cheap to do).

Other than that, all looks really good! Congratulations @gkasprow

[marmeladapk]

From @hartytp on 2018-02-02 18:25

NB haven't bothered to check performance without the PLL. We may not quite hit the noise floor of the DDS in that case, but clearly there is no drastic problem.

[marmeladapk]

From @jbqubit on 2018-02-06 19:31

Why did @WeiDaZhang switch to higher phase noise E4422B Ref vs Wenzel?

Looks like <5 dB deviation from Datasheet at single frequency. Nice job! Is it similar at other output frequencies?

[marmeladapk]

From @hartytp on 2018-03-16 20:15

Before I forget: @cjbe tested phase synchronisation with FPGA. All looks good!

[marmeladapk]

From @jordens on 2018-03-16 20:36

Nice. @cjbe @hartytp with the FPGA driving SYNC? Or just among the DDS?

[marmeladapk]

From @cjbe on 2018-03-16 21:37

@jordens with FPGA driving sync. I tested synchronisation between channels on the same board and channels on different boards.

The DDS sync signal was generated by a TTLClkGen, and IO_UPDATE by a TTL SERDES output. I adjusted the sync input delay on each channel to sit in the centre of the eye - thus the SYNC_CLK of each DDS is aligned to better than 1 DDS clk period (=1ns @ 1GHz).

I choose an offset between IO_UPDATE and the sync TTLClkGen such that IO_UPDATE aligned roughly on the falling edge of the DDS SYNC_CLK. (I did not look at the CPLD timing report to work out if this would work with a fixed value over PVT).

I saw no synchronisation errors over ~10^10 IO_UPDATES, but I did see at least one SYNC_SMP_ERR from most of the AD9910 over this 10^10 events using the standard 300ps validation window - I have not looked to see if this happens with slightly smaller validation windows.

[marmeladapk]

From @jordens on 2018-03-16 21:52

Ok. Nice!

[marmeladapk]

From @hartytp on 2018-03-17 10:58

(I did not look at the CPLD timing report to work out if this would work with a fixed value over PVT).

This still concerns me, and is something we need to look at. @jordens do you know how well constrained this timing is atm?

[marmeladapk]

From @jordens on 2018-03-17 12:07

SYNC is not routed through the CPLD. That can't be related to SYNC_SMP_ERRs. How are you checking for IO_UPDATE synchronization errors? I don't know about the constraints there (other than the 7.7ns nominal propagation delay) and whether that's even specified.

[marmeladapk]

From @cjbe on 2018-03-17 12:44

There are two considerations here:

1. SYNC_SMP_ERRs:
   This is just due to the jitter of the FPGA generated SYNC relative to the AD9910 1GHz clock. I see a narrower eye using the FPGA generated SYNC vs the AD9910 sync (~6 taps = ~450ps for the FPGA generated clock). I am using a validation window of 300ps, so it is not surprising that at least 1 in 10^10 events closes the eye to <300ps, but may be fine for a validation window one tap smaller.

2. IO_UPDATE sync errors:
   I am setting CFR1 bit 11 so the phase resets on IO_UPDATE, I am then triggering a scope on a signal synchronous with IO_UPDATE, and looking at the RF phase relative to this trigger with a scope on persist. As the RTIO fine clock is 1ns, and the SYNC_CLK period is 4ns, there are 4 possible IO_UPDATE alignments. With one of these I see frequent phase alignment errors on the scope, but with the other 3 I have not seen any - I choose the best out of these three by looking at the IO_UPDATE timing at the AD9910.

My only worry is how large the PVT change in the 7.7ns propagation delay is - if it is larger than ~2ns (which is probably is) things are getting marginal (the IO_UPDATE setup/hold is 1.75ns/0ns).
We could register IO_UPDATE on the DDS0_SYNC_CLK at the CPLD - from the CPLD datasheet it looks like the PVT on the clock input buffer and standard output buffer sum to ~0.9ns, so this may well work out.

[marmeladapk]

From @jordens on 2018-03-17 15:44

That's what I thought. I don't think P is relevant for us (you'll have to tweak it anyway). And I don't think VT are more than 500 ps for our VT variability.
Registering io update doesn't help AFAICT because (1) there is no way to delay the cpld output to meet sync clk s/h and (2) the s/h window on the cpld is unlikely to be significantly larger than the 2.25 ns of the DDS.

[marmeladapk]

From @hartytp on 2018-03-21 11:46

NB we have an IO_UPDATE return on EEM1 in the current design, so if we see any PVT issues with the phase synchronisation, we should be able to use that to calibrate the delays in the CPLD buffers. Given that, I no longer have any concerns about phase synchronisation with Urukul!

[marmeladapk]

From @jordens on 2018-04-27 17:01

The warmest chip is the clock fanout. With just convective cooling it is ~75C wenn flat on a table.

Pulled fresh out of a crate after thermalizing with convective cooling upright the DDS also get into the 70C.

[marmeladapk]

From @hartytp on 2018-04-27 18:34

What do people think about maybe sprinkling a few cheap, low-profile SMT/THT heatsinks on Urukul (and maybe/other EEMs with significant power consumption) near the most power-hungry ICS?

[marmeladapk]

From @jordens on 2018-04-27 18:48

I think it's ok. I'm just surprised the fanout gets so hot compared to the DDS.

[marmeladapk]

From @gkasprow on 2018-04-27 19:45

THere is a place under on the other side for small ceramic heatsink

[jordens]

@WeiDaZhang @hartytp do you by chance have Urukul/v1.[123] (with the new loop filter component values) phase noise data (like #3 (comment))?
And does anybody have a phase noise spectrum with Urukul clocked from Kasli?

[hartytp]

@jordens IIRC we posted something a while back for a version of Urukul (v1.1?) that @WeiDaZhang used to prototype the new loop filter, but it wasn't clocked from Kasli.

If you need this measurement we can probably make it. @WeiDaZhang

[jordens]

Found your measurement with SMA clock: https://github.com/sinara-hw/sinara/issues/504#issuecomment-366386549

[jordens]

And sinara-hw/sinara#515 (comment) has the bare Kasli Si5324 data. @WeiDaZhang / @hartytp do you remember which Si5324 PLL parameters were those?

[hartytp]

@jordens any objection to closing this issue now?

[jordens]

Yeah. Should have been split much earlier.