Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

DRTIO clock recovery/Si5324 in Sinara #515

Closed
hartytp opened this issue Mar 1, 2018 · 111 comments
Closed

DRTIO clock recovery/Si5324 in Sinara #515

hartytp opened this issue Mar 1, 2018 · 111 comments

Comments

@hartytp
Copy link
Collaborator

hartytp commented Mar 1, 2018

@cjbe @WeiDaZhang and I have been looking at the timing stability of DRTIO on Kasli (all comments apply equally to Sayma of course).

Expectation:
E1. The latency between TTLs on Kasli DRTIO slaves and TTLs on the master should be deterministic and fixed between power cycles. Equivalently, the phase relationship between the master clock, measured at the SMA input), and the recovered RTIO clock, measured at the MMCX on the slave, should be fixed between power cycles.
E2. The latency between TTLs on Kasli DRTIO slaves and TTLs on the master (or, equivalenty, the phase relationship between the two clocks) should be stable to <<1ns over standard operating temperature range (say, something like 10C-20C).
E3. DRTIO is a basic piece of ARTIQ infrastructure, and the better it is (within reason) the more uses it is likely to find. Having a high-quality way of distributing time (e.g. clocks for data converters/TDCs etc) over fibres with good stability is a high-value feature for ARTIQ, so we should try to make the performance "as good as reasonably possible" (without overcomplicating our designs). i.e. let's not shoot for something that just barely meets our most lax specification if there is a way of doing significantly better without too much additional cost/complexity.

Observations:
O1. In @cjbe's measurements, the latency was not constant between power cycles. As @jordens has independently pointed out, this appears to be expected for the Si523, data sheet says Input to output phase skew after an ICAL is not controlled and can assume any value.
O2. The phase of clock output on the slave is extremely unstable over temperature. Touching the Xtal makes the slave clock move w.r.t. the master by multiple cycles of the 150MHz clock. The phase also wanders around noticeably even when Kasli is left alone in standard lab conditions and running at constant duty cycle etc.

Discussion:
AFAICT, DRTIO uses the Si5324 as follows:

  • At startup, the Si5324 "free run" mode is used, which internally routes the 114.285MHz clock (generated from the Xtal on Kasli) to the clockin_2 input on the Si5324. This is used as a reference to generate the 125MHz (or 150MHz on Sayma) RTIO clock to establish the DRTIO link.
  • Once that link is established, the transceivers produce a 125MHz/150MHz clock, which is routed to the Si5324 clockin 1.
  • The Si5324 then performs "hitless switching" by switching from clockin2 to clockin1 without changing PLL settings. This is possible because the two inputs have independent dividers, which can be set to large values. By setting these dividers to around 10,000 the two clocks are divider down to the same frequency of around 16kHz, which is the frequency that the Si5324 PFD runs at to generate the RTIO clock.
  • Note also, that the way the Si5324 works is that it locks a microwave VCO (around 5GHz) to the on-board Xtal with a fairly wide bandwidth. It then locks the VCO output to the reference with a lower bandwidth (<1kHz) by tuning the fractional part of the divider. Since the Si5324 locks are implemented digitally, one might think this would be done using a high-order loop filter to aggressively reject in band (slow) fluctuations in the Xtal. Based on my measurements, this seems not to be the case; the Si5324 does not follow the reference well even for frequencies quite far below BW (I assume it's a standard 2nd-order PLL).

Comments:
C1. Since we are very sensitive to the performance of the Xtal/XO on Kasli, we should be using a high-quality one. Kasli currently has a very cheap Xtal. We replaced this with a high-quality Crystek 125MHz XO (different package, dead-bugged on) and found that the phase stability of the Si5324 clock was significantly improved -- although the phase fluctuations of the 150MHz clocks were still large and easily notable by eye on a scope.
C2. Running the PLL at such a low frequency is unlikely to be a good idea from a noise/stability perspective. Using the 125MHz cyrstal, we were able to run the PFD at a higher frequency (around 2MHz IIRC). This made a significant improvement in the phase stability although, again, not enough to solve the problems. We should aim to run the PFD in our "jitter cleaner" loop at the highest frequency possible to minimize noise/drift.
C3. The Si5324 recommends using a reference which is not related by a rational number to our reference/output clocks. AFAICT, that's to minimize in-band-spurs. AFAICT, that recommendation is not good advice for us, since the problems introduced by running the PFD at a really low frequency are much worse than a few spurs (which are generally rejected by subsequent PLLs anyway).
C4. As @sbourdeauducq has pointed out, it may be possible to fix the phase non-determinism of the Si5324 by wrapping it in an additional PLL using the FPGA (although, that somewhat defeats the purpose of having the Si5324 in the first place).

Recommendations for DRTIO v2.0:
R1. Replace the Si5324 with a proper PLL! This could either be an analog PLL or a digital one like WR. The optimal choice will depend a bit on the measured noise and stability of the CDR clock from the FPGA. We'll aim to measure that soon.
R2. Use a TCVCXO for the flywheel oscillator. The best TC(VC)XOs are generally at around 10MHz-25MHz, so we should probably follow WR and use a decent 25MHz TCVCXO for the flywheel oscillator.
R3. Run the PLL PFD directly at 25MHz rather than some low frequency
R4. Use a reasonably high-order loop filter (we used 3rd order for exactly this reason on the clock mezzanine that Weida and I designed).
R5. If the TCVCXO is well-enough specified, we may be able to do our "hitless switching" by just setting the control voltage to mid-range during link initialisation (someone would need to check the numbers). That simplifies things a lot.

@hartytp
Copy link
Collaborator Author

hartytp commented Mar 1, 2018

@dtcallcock I remember Jeff saying he'd measured timing stability for WR. Would you mind asking him for any data he has on this, please?

@hartytp
Copy link
Collaborator Author

hartytp commented Mar 1, 2018

PS sorry for the essay, wanted to get my thoughts clear while this is fresh in my mind!

@sbourdeauducq
Copy link
Member

As @sbourdeauducq has pointed out, it may be possible to fix the phase non-determinism of the Si5324 by wrapping it in an additional PLL using the FPGA (although, that somewhat defeats the purpose of having the Si5324 in the first place).

Not necessarily: the FPGA PLL output would only drive the Si5324 input, and you still get the jitter filtered (the clock that is used for the rest of the design is the Si5324 output only).

I see three ways of doing this:

  • put the Si5324 in a FPGA PLL feedback path, with the PLL reference being the recovered clock, and the result being the Si5324 output that the PLL aligns with the reference. This is quite risky as the PLL is likely to become unstable due to the slow response time of the Si5324, but if it works, it might give the best deterministic skew performance. Someone with better knowledge of control theory than I do could perhaps see if this hack has a chance to work - it's just a vague idea so far.
  • detect the skew with the FPGA (e.g. similar procedure as DDR3 write leveling) and correct using a MMCM with a fine phase shift inserted between the recovered clock and the Si5324 input. Deterministic skew performance is limited by the resolution of the phase shift (1/56 of VCO frequency, the VCO having a maximum of 1200MHz, so >15ps).
  • detect the skew with the FPGA and correct with a ODELAY. There are no ODELAYs on Artix-7 and this would be limited by the ODELAY tap resolution (on Kintex Ultrascale: 2.5ps to 15ps depending on PVT).

I'm surprised we didn't see the Si5324 problems before, when we tested this: https://github.com/m-labs/drtio_transceiver_test

@sbourdeauducq
Copy link
Member

sbourdeauducq commented Mar 1, 2018

The phase of clock output on the slave is extremely unstable over temperature. Touching the Xtal makes the slave clock move w.r.t. the master by multiple cycles of the 150MHz clock. The phase also wanders around noticeably even when Kasli is left alone in standard lab conditions and running at constant duty cycle

I wonder if this is normal. Wouldn't that also cause problems in typical situations where the Si5324 is used with a shallow elastic buffer?

This is also quite difficult or impossible to fix with the FPGA.

@hartytp
Copy link
Collaborator Author

hartytp commented Mar 1, 2018

Unsurprisingly, WR is really nicely designed and very carefully thought through. They have lots of good application notes on this, which contain a lot of the info we need (GTX noise measurements etc). I started pooling some of the data on the Wiki https://github.com/m-labs/sinara/wiki/SinaraClocking

The good news is that they're able to recover a really pretty good clock over 1 3km fibre, even after FPGA transceivers. -105dBc/Hz at 10Hz (for reference the SRS rubidium clock is -105dBm/Hz at 1Hz, -135dBc/Hz at 10Hz).

@hartytp
Copy link
Collaborator Author

hartytp commented Mar 1, 2018

So, it all comes down to designing a good PLL with a good flywheel oscillator. That's not too hard.

@hartytp
Copy link
Collaborator Author

hartytp commented Mar 1, 2018

detect the skew with the FPGA (e.g. similar procedure as DDR3 write leveling) and correct using a MMCM with a fine phase shift inserted between the recovered clock and the Si5324 input. Deterministic skew performance is limited by the resolution of the phase shift (1/56 of VCO frequency, the VCO having a maximum of 1200MHz, so >15ps).

I suspect that the stability/noise of this approach will not be great. But, could work as a bit of a hack to get the current HW up and running while we design a better long-term solution.

@sbourdeauducq
Copy link
Member

To get the current hardware running, perhaps dropping in a Si5326 will work. Double-check the datasheet, but it seems: it is pin-compatible, it has mostly the same register interface, it has input-to-output skew control, and its features generally are a superset of the 5324's.

@gkasprow
Copy link
Member

gkasprow commented Mar 1, 2018

there is one possible fix - add guard shield around the oscillator. It is not written in Si5324 datasheet, but is recommended for some other Silabs chips with similar oscillator.
It is simple - dedicated PCB polygon around the crystal, dedicated copper area below it, connected to the GND in one place, close to SIlabs XI/XO and nowhere else.

@hartytp
Copy link
Collaborator Author

hartytp commented Mar 2, 2018

@gkasprow what does that do? Just an EMI shield? These problems are mainly thermal. (A screening can to prevent air currents reaching the crystal would probably help a bit however.

@dtcallcock
Copy link
Member

Here's Jeff Sherman's poster on some measurements he did at NIST on the SevenSols WR-LEN. sherman_WSTS2017_poster.pdf

He got 100fs because he's got mad skillz.

@gkasprow
Copy link
Member

gkasprow commented Mar 4, 2018

@hartytp Yes, I meant EMI shield.

@hartytp
Copy link
Collaborator Author

hartytp commented Mar 5, 2018

Thanks David and Jeff! Mad skills indeed.

So, Jeff thought that the SevenSols WR-LEN might be implemented in a slightly different way to the original WR and might be somewhat better. Anyone know any details about its implementation? AFAICT it's not open source, or am I missing something?

@hartytp
Copy link
Collaborator Author

hartytp commented Mar 5, 2018

There was a mistake in the measurements reported above. @cjbe retook the data more carefully and found:

  • Master and slave Si5324 both running at 2MHz PFD frequency and max loop BW of around 500Hz
  • One of Master/slave has the default Xtal, one has a nice 125MHz VCXO.
  • Startup clock is a synth + power splitter provided to both master and slave during initialisation (disconnected from master + slave once DRTIO is up).
  • Measure timing stability by putting the slave and master Si5324 outputs onto a fast scope and looking at the pk-pk phase deviation over a couple of days.
  • Found stability of 50ps pk-pk, which seems to be consistent with the noise floor of the measurement.

Once we receive more Kasli, I'll repeat this measurement using an interferometer (mixer) to give a sub ps measurement noise floor.

@hartytp
Copy link
Collaborator Author

hartytp commented Mar 5, 2018

So, it seems that the current hardware can provide <<ns timing stability with correct Si5324 settings.

Note that the phase of the Si5324 is still random at startup. However, this should be fixable by using a PLL in the FPGA to add a phase shift to the Si5324 input clock (this may degrade the phase stability a little however). The FPGA can then be used to measure the Si5324 startup phase and tune the Si5324 input phase shift appropriately. So, we should be able to get a decent solution with current hardware and only gateware/firmware changes.

@hartytp
Copy link
Collaborator Author

hartytp commented Mar 5, 2018

Pending the result of the interferometer measurement to look at the drift of the Si5324 phase, I'd still like to suggest that we should consider using a solution based on a proper low-noise PLL IC with a better reference oscillator (VCO).

My thinking here is that if we can get ps-level timing stability for the recovered clock -- and Jeff's data suggests that we should be able to -- then there are a lot of cool things we can do with it.

  • clock for all data converters/TDCs etc from the recovered clock simplifies wiring a lot
  • For Sayma, we can simplify the firmware/gateware by only supporting the recovered clock as an option (once it has been verified to provide sufficient performance). As @sbourdeauducq pointed out, this could make synchronising DACs to the RTIO clock a lot easier.

If ps timing stability can be achieved with the Si5324 + Xtal then great, otherwise, I'd like to prototype a better solution with the intention of applying it to Kasli/Sayma/Metlino once it's been well tested using current hardware.

@hartytp
Copy link
Collaborator Author

hartytp commented Mar 5, 2018

To do the design, the one thing we need to finalize is the choice of RTIO frequencie(s).

  • Currently, we're using 125MHz and 150MHz. I'd like to propose that we pick 125MHz as the only supported RTIO frequency.
    • 150MHz is obnoxious (doesn't give integer numbers of ns, can't use AD9910 at 1GHz etc)
    • We only have 150MHz because of the chosen Sayma SAWG parameters. However, this seems backwards: RTIO frequency is a global system property, so the data converters should work with the chosen RTIO frequency, not the other way around. It's obviously not scalable to allow a converter to select the RTIO frequency (what happens when we design a new fast data converter board? Do we allow that to pick another RTIO frequency again?)
    • I still think that the 1GHz data rate, 2GHz DAC clock, 125MHz RTIO clock is the most appropriate configuration for most ion trapping experiments. The loss in base-band bandwidth (i.e. the separation between the two tones produced by the SAWG) isn't generally an issue since +-125MHz of IQ BW is fine for all ion trapping experiments I can think of as: it's more than the BW of any AOM; it's more than typical motional frequencies/Zeeman splittings.

The RTIO frequency affects our PLL design:

  • With a fixed RTIO frequency, we can choose a really nice VCO to run directly at the RTIO frequency.
  • With a flexible RTIO frequency, we need to run the VCO at a lower frequency (e.g. 25MHz). This has implications for the choice of VCO we use (the best quality ones are only available at certain frequencies) and also means that the PLL output is lower than the RTIO frequency. As a result, we need an additional PLL to boost it (NB going from 25MHz to 2GHz in a single PLL for Sayma is a very large frequency change and will not give great phase noise).

Any objections to fixing the RTIO frequency to 125MHz if that makes the PLL design easier/better (obviously, other frequencies available as population options, potentially with somewhat worse performance)? @jordens

@jordens
Copy link
Member

jordens commented Mar 5, 2018

The reasons to choose 150 MHz f_RTIO and my take on the arguments you bring up are described elsewhere. I don't think rehashing them is necessary as they haven't changed. There would probably be quite some development needed to go to 1 GHz SAWG now but that is also a chance to revise the parametrization and datapath design and rethink the specs. The current and potential SAWG users would need to think hard and weigh in.

@dtcallcock
Copy link
Member

So, Jeff thought that the SevenSols WR-LEN might be implemented in a slightly different way to the original WR and might be somewhat better. Anyone know any details about its implementation? AFAICT it's not open source, or am I missing something?

I opened one up and the important chips seem to be:

Artix-7 XC7A35T
Analog Devices AD9516-4BCPZ
Micrel KSZ9031RNXCA
SiLabs Si570 BBC000121G
VCXO 25.000 SRe647A (unknown vendor)

Jeff claims it's a development of what was at some point an open source project. I will ask him for more details.

@jordens
Copy link
Member

jordens commented Mar 5, 2018

@gkasprow
Copy link
Member

gkasprow commented Mar 5, 2018

@dtcallcock The WR-LEN is not open source hardware but they keep the gateware open since it is based on original WR development. However the chipset is similar as SPEC WR node I developed for CERN a few years ago.
The main difference is that they use ZynQ SoC instead of original Spartan FPGA and they use Silabs I2C tuned crystal oscillator instead of original DAC + VCXO + PLL with VCO.
However the DMTD still uses same 25MHz helper VCXO oscillator.
The PN is LF VCXO026156.
Original design used 25MHz VCXO + external x5 PLL because the price and availability of the 125MHz VCXO was a barrier. Later on 62.5 MHz instead of 125MHz was used to clock the WR core.
Later on in order to improve the jitter of the WR node, which was dominated by the jitter of the CDCM61004, Silabs Si570 chip was used instead of the VCXO + PLL. This required significant modification of the WR node gateware because I2C controller had to be included into the loop.
So probably 7Sols went the same way and in this way improved the jitter performance.
@twlostow knows more details about this development.

@gkasprow
Copy link
Member

gkasprow commented Mar 5, 2018

Edit: they say that use VCXO and TCXO controlled by DAC, so there should be yet another oscillator. Silabs is probably used for general purposes clocking

@gkasprow
Copy link
Member

gkasprow commented Mar 5, 2018

@dtcallcock in what bandwidth Sherman got his 100fs?

@dtcallcock
Copy link
Member

@gkasprow From the poster is seems he was comparing the master input 10MHz with the slave output 10MHz with a 50Hz effective BW and averaging down for 10^4s to get to 100fs. This was in a temperature and humidity controlled chamber. If you need more specifics I can ask him or just give you his email.

@gkasprow
Copy link
Member

gkasprow commented Mar 5, 2018

For WR-ZEN design they use the same VCXO as we use for Urukul (CVHD-950) + low jitter divider (some chip from NS) to get 10MHz. Then they apply intensive filtering using 2x MCL to get rid of higher harmonics. So they do probably the same for WR-LEN.
So the main difference to the original SPEC design is better VCXO with lower phase noise, lack of the PLL and VCO. Control circuit is the same since I see two DACs + reference used in SPEC design

@hartytp
Copy link
Collaborator Author

hartytp commented Mar 5, 2018

The reasons to choose 150 MHz f_RTIO and my take on the arguments you bring up are described elsewhere. I don't think rehashing them is necessary as they haven't changed. There would probably be quite some development needed to go to 1 GHz SAWG now but that is also a chance to revise the parametrization and datapath design and rethink the specs. The current and potential SAWG users would need to think hard and weigh in.

We're comparing two configurations:

  • 600MSPs DAC data rate (300MHz nyquist), with the DAC clock up to 2.4GHz, RTIO frequency at 150MHz (x4 parallelisation in the CORDICS to generate 600MSPS data)
  • 1GSPs DAC data rate (500 MHZ nyquist) wit the DAC clock up to 2GHz, RTIO frequency at 125Hz (x8 parallelisation in the CORDICS to generate 1GSPS Data).

@jordens I'm going to try to summarise the arguments here. Please correct me if I'm wrong:

  1. Carrier bandwidth: 1GHz DAC clock (125MHz RTIO clock) obviously gives a higher carrier bandwidth than 600MHz by nearly a factor of two. For single-tone operation, this directly determines the bandwidth of the signal one can produce.
  2. Baseband bandwidth: Ignoring a small correction due to the anti-aliasing filters, the baseband (IQ) bandwidth one can get out of the SAWG in two-tone operation is the +-f_RTIO. So, operating at 600MSPs gives one +-150MHz baseband bandwidth, compared with +-125MHz bandwidth for the 1GSPSscase. This bandwidth effectively determines the maximum separation between the tones (not the maximum RF frequency) in two-tone mode.
  3. Compile-time: the compile time is a super-linear function of the CORDIC parallelisation factor, so the 1GSPs case will be quite a bit slower to compile than the 600MSPs case. This will have a nock on effect on the complexity/cost of development for the two cases.
  4. Yak shaving: in principle switching from 600MSPs to 1GSPs just needs a few settings changed. In practice, it could take some development time (and further funding?).

IIRC, we've agreed to start with the 600MSPs case as the quickest way to get going. However, I'm keen to switch to the 1GSPS operation in the long run, as this maps better to my use-cases (and, AFAICT, to most ion trap use cases).

So, question for other potential Sayma users (@jbqubit @dhslichter @dtcallcock @cjbe) how do you plan to use Sayma? Do you want/need the 600MSPs (150MHz RTIO) use-case? Or, like me, would you prefer the full 1GSPs bandwidth of the DACs?

@dtcallcock
Copy link
Member

@hartytp

1GSPs DAC data rate (500 MHZ nyquist) wit the DAC clock up to 2GHz, RTIO frequency at 150Hz (x4 parallelisation in the CORDICS to generate 1GSPS Data).

Should that be '...125MHz (x8 parallelisation...'?

@hartytp
Copy link
Collaborator Author

hartytp commented Aug 28, 2018

@gkasprow The Si549 and LMK61e07 are the only two DCXOs of this quality that I'm aware of, are you aware of any others?

@gkasprow
Copy link
Member

What about limiting the I2C noise by adding capacitors that reduce the edges?
Silabs VCXOs are quite expensive and their availability is very limited. I.e. Si570/571 need to be ordered 2 months in advance. DAC + VCXO costs the same and solves issues with I2C noise.
CERN develops high-performance WR receiver suitable for RF applications. Tom is using there DOT050V VCXO + AD5662

@sbourdeauducq
Copy link
Member

Silabs VCXOs are quite expensive

While there is indeed no stock, the Si549 is $16 at Arrow. The dollar amount isn't such a high price to pay if that avoids some horrible silicon bugs and workarounds like those I2C caps. I'm definitely not looking forward to another HMC7043-style chip.

@WeiDaZhang
Copy link
Collaborator

@gkasprow I tried to add capacitors and using different power sources on LMK61E07. The effectiveness is hardly noticeable. My feeling was the i2c noise is inner coupled to its output, rather than SI/PI issue.

@sbourdeauducq
Copy link
Member

@WeiDaZhang I think Greg was talking about adding capacitors to SCL and SDA to increase rise and fall times. Depending where the noise coupling happens inside the chip, that may or may not help.

@hartytp
Copy link
Collaborator Author

hartytp commented Aug 29, 2018

What about limiting the I2C noise by adding capacitors that reduce the edges?

@gkasprow I don't think that will help. Firstly, the noise we really care about is the stuff below about 100kHz. However, we can't reduce the edge rise time to >>10us because we need to operate the I2C bus quite fast to get the update rate required for closed-loop operation.

My guess is that this noise originates internally to the DCXO, so we're stuck with it.

Having said that, it might be worth sending the data we have to Ti and asking them about it. @WeiDaZhang are you happy to post something on the TI engineering forums?

@WeiDaZhang
Copy link
Collaborator

to post something on the TI engineering forums?
@hartytp will do.

@WeiDaZhang
Copy link
Collaborator

WeiDaZhang commented Sep 3, 2018

TI suspects that my hand soldering may have damaged the chip.
https://e2e.ti.com/support/clock_and_timing/f/48/t/723417
I've ordered the TI eval board this time which has got the chip (LMK61E2) soldered on it.
We will try again when it arrives.

@hartytp
Copy link
Collaborator Author

hartytp commented Sep 3, 2018

TI suspects that my hand soldering may have damaged the chip.

I'll be surprised if that's really the case, but using the TI eval board does sound like a good plan -- it would be nice to have a second option besides the Si549.

@hartytp
Copy link
Collaborator Author

hartytp commented Sep 24, 2018

Initial phase stability using a fast scope:

bitmap

  • 2x KC705s both locked to a common 100MHz reference
  • using clocker as an active power splitter (75fs/K temp co for the clock buffer IC)
  • scope ch1 + ch2: GTX clock output from KC705 2, using a passive power splitter
  • scope channel3 + channel4: are the outputs of the two Si549s (these are the blocks labeled "main")
  • we measure approx 40ps RMS jitter between all channels over a period of hours, limited by the scope. This suggests that the stability of the DDMTD loop is <<40ps, probably in the low single-digit ps.

hhhhh

@hartytp
Copy link
Collaborator Author

hartytp commented Sep 24, 2018

How good is good enough? Depends on the use-case, but a common one would be producing RF at around 300MHz (maybe a bit higher, but that makes the maths nice!). Generally, the target for QIP would be phase stability better than 1deg, with 0.1deg being ideal, measured over say 24 hours ish. 1deg=10ps. So, we're looking for stability better than 10ps, and ideally around 1ps (0.1deg).

For direct microwave production at 3GHz (e.g. calcium) assuming we want to generate our LO from the WR-distributed reference, that goes to 0.1ps-1ps, which is much more challenging. Not clear if we can hope to do that via WR...

Still, these results show that the WR PLL noise + drift, measured under realistic conditions, is good enough to clock a RF module like Sayma from.

@gkasprow
Copy link
Member

gkasprow commented Sep 24, 2018

There is recent development on low jitter version of WR switch. Some people are also experimenting with better oscillators, but for us it would be an overkill.

@hartytp
Copy link
Collaborator Author

hartytp commented Sep 24, 2018

@gkasprow thanks for the cool links.

This is a solution with a substantial lower cost than using multiple H-masers.

That's a pretty low bar :)

From my end, the idea was to do as well as we can with of order $20 USD of parts and about 1cm^2 of board space, so the Wenzel oscillator is not an option.

I'd be interested to see the phase noise plot for that, but my guess is that it's heavily into the regime of decreasing returns in terms of dBc/$$$.

Cool project though.

If you compare the specifications for the WR low jitter VCXO with the Si549, they are basically identical. Except the Si549 performs better at high frequencies, which is to be expected since it has a higher output frequency.

And then you have to take into account the fact that using a VCXO + DAC you have to watch out carefully for any analog noise associated with your DAC/reference/pickup/ground loops/etc, where as the Si549 takes care of that all internally. So, I'll be surprised if their results are actually as good as ours once the PLL is locked. (Not to mention the fact that, IIRC, the PLL used by WR is only second-order so there is quite a lot of noise feed-through from the DDMTD at middling offset frequencies. In contrast we're using a 3rd order filter which gives much nicer performance).

Basically, it's nice, but I think our way will work better ;)

@hartytp
Copy link
Collaborator Author

hartytp commented Sep 24, 2018

Except the Si549 performs better at high frequencies, which is to be expected since it has a higher output frequency.

@gkasprow why do they use 25MHz VCXOs instead of a 125MHz oscillator? I never understood that. Is it to do with the frequencies where the best VCXOs are available, or some other constraint?

@hartytp
Copy link
Collaborator Author

hartytp commented Sep 24, 2018

Basically, it's nice, but I think our way will work better ;)

I'd be interested to hear what they think of it in case I've missed anything!

@gkasprow
Copy link
Member

@hartytp This design choice was taken long time ago, around 2008.
We needed clock buffer and at that time price of 125MHz VCXOs was prohibitive so it was a compromise to go for 25MHz one and the PLL with fanout. At that time nobody played with SI570 and we were not sure if this would work due to its complicated programming. Si549 was not yet available. Once the design was working reliably, it was copied to multiple designs.

@hartytp
Copy link
Collaborator Author

hartytp commented Sep 24, 2018

The final comment I'll make on this while I'm nerding out on time distribution is that the Si-Labs got the idea right with the Si549: use an XO + frac-n pll instead of a VCXO. XOs give way better performance for a given cost point than VCXOs. Plus, you scrap all the hard analog bits associated with a low-frequency PLL and get a much cleaner system.

@hartytp
Copy link
Collaborator Author

hartytp commented Sep 24, 2018

@gkasprow that makes sense. Well, it'll be interesting to see how well this works out on Kasli.

@gkasprow
Copy link
Member

AFAIR we tested such fractional PLL in CBM clock synchronisation and there were some issues with Silabs chips. I will ask my coleague about it since he was playing with firmware at the time.

@hartytp
Copy link
Collaborator Author

hartytp commented Sep 24, 2018

@gkasprow let me know what he says, because we haven't found any issues so far with the Si549. I wonder if they were using some other chip?

@gkasprow
Copy link
Member

We were using Si5338

@gkasprow
Copy link
Member

For CBM experiment we built clock synchronisation using AFCK and SI570, but AFAIR no detailed jitter measurements were done. Our PhD student was implementing WR node on SI570 in similar way that you want to do it with Si549

@hartytp
Copy link
Collaborator Author

hartytp commented Sep 24, 2018

Our PhD student was implementing WR node on SI570 in similar way that you want to do it with Si549

How far did he get?

We looked at the Si570 when we started out. I can't remember why we decided it wasn't an option. The noise is worse that the 549, but I think there was some other reason as well. FWIW, I think the two are pin/footprint compatible, and have very similar programming interfaces, so it's easy to swap them.

We were using Si5338

Wow! That's a lot more complex than the 549/570. I'd have to read the data sheet quite carefully to figure out whether our scheme would work with that.

@gkasprow
Copy link
Member

@mguminski Can you write a few words about your results?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

7 participants