WIDAR Correlator

by Stephan W. Witz last modified Jun 22, 2017 by Emmanuel Momjian

Introduction

The correlator configurations offered for general observing may be divided into three basic modes: wideband, spectral line, and subarrays. The possible setups are also subject to the integration time and data rate restrictions outlined in the section on Time Resolution and Data Rates. The possibilities and restrictions are embodied in the General Observing Setup Tool (GOST and in the Resources section of the Proposal Submission Tool (PST), which must be used to define the correlator configuration for General Observing (GO) and Shared Risk Observing (SRO) proposals.

Note that for general observing phased array configurations are only allowed as part of VLBI experiments (see the section on VLBI Observations); other uses such as pulsar observations must be done as Resident Shared Risk Observations.

Wideband and spectral line observing modes with the WIDAR correlator are described below. For the subarray mode, we refer to the Subarrays section of the OSS.

For technical details about the WIDAR correlator, refer to References 14 in Documentation.

 

WIDAR Correlator: Wideband Observing

The wideband observing setups provide the widest possible bandwidth for a given observing band, with channel spacing depending on the number of polarization products as listed in the following table 4.1.1:

 

Table 4.1.1: Wideband & Subarray Correlator Options
(all but P and L-bands)
Polarization productsChannel spacing
Full (RR, RL, LR, LL) 2 MHz
Dual (RR and LL) 1 MHz
Single (RR or LL) 0.5 MHz

 

8-bit wideband setups are available for all observing bands, providing a total of 2 GHz of bandwidth per polarization (1 GHz per polarization at L-band, and 256 MHz per polarization at P-band). 3-bit setups are available for all bands above S-band, providing total bandwidths per polarization of 4 GHz (C/X-bands), 6 GHz (Ku-band), or 8 GHz (K/Ka/Q-bands). In all cases, except for P and L-band, each of the subbands is 128 MHz wide. At L-band the default is 64 MHz/subband, yielding channels twice as narrow as those listed in the table above, while at P-band the default is 16 MHz/subband, resulting in 125 kHz channel spacing.

In many frequency bands, the total processed bandwidth is less than that delivered by the front-end. In those cases, the observer may independently tune two 1 GHz baseband pairs when using the 8-bit samplers, or four 2 GHz baseband pairs when using the 3-bit samplers, or choose to have a mix 8-bit and 3-bit samplers. The tuning restrictions are described in the section on VLA Frequency Bands and Tunability, and the 8-bit and 3-bit samplers are described in the section on VLA Samplers.

 

WIDAR Correlator: Spectral Line Observing



Basebands and Subbands

Observers have access to very flexible correlator configurations using up to 64 subbands in up to 4 basebands sampled with the 8-bit and/or the 3-bit samplers. These capabilities may be summarized as follows:

  • Two 1 GHz baseband pairs using the 8-bit samplers, or four 2 GHz baseband pairs using the 3-bit samplers, independently tunable within the limits outlined in the section on VLA Frequency Bands and Tunability. The 8-bit baseband pairs are referred to as A0/C0 and B0/D0, while the 3-bit samplers are A1/C1, A2/C2, B1/D1, and B2/D2. The AC/BD nomenclature corresponds to that of the IF pairs in the pre-expansion VLA.
  • Up to 16 subband pairs (spectral windows) in each 3-bit baseband pair, and up to 32 subbands in each 8-bit baseband pair, for a total of up to 64 subbands in any correlator configuration:
    • Tuning, bandwidth, number of polarization products, and number of channels can be selected independently for each subband;
    • All subbands must share the same integration time;
    • No part of a subband can cross a 128 MHz boundary;
    • Subband bandwidths can be 128, 64, 32, …, 0.03125 MHz (128 / 2n, n=0, 1, …, 12).
  • The sum over subbands of channels times polarization products is limited to 16384 (without recirculation):
    • These may be spread flexibly over subbands and polarization products, in multiples of 64: 64, 128, 192, 256, 384, …, 16384 cross-correlation products;
    • Recirculation may be used to increase the number of channels per subband for subbands narrower than 128 MHz. Baseline Board stacking may be used to increase the number of channels per subband for setups requiring less than 64 subbands;
    • Assigning many channels to a given subband may reduce the total bandwidth and/or the total number of subbands available.

The remainder of this section discusses the various limitations in more detail, including some examples to show how they come up in practice.

 

Subband Tuning Restrictions

Each subband may be placed anywhere within a baseband, with the caveat that no subband may cross a 128 MHz boundary. Mathematically:

νBB0 + n×128 MHz <= νsbLow <= νsbHigh <= νBB0 + (n+1)×128 MHz

where:

νBB0 the lower frequency edge of the baseband;
n= 0, 1, …, 7 (, …, 15) (any integer between 0 and 7 for 8-bit, between 0 and 15 for 3-bit);
νsbLow
the lower edge of the subband
(the subband center frequency minus half the subband bandwidth);
νsbHigh
the upper edge of the subband
(the subband center frequency plus half the subband bandwidth).

For example, if the baseband were tuned to cover 10000–11024 MHz, one could place a 64 MHz subband to cover 10570–10634 MHz, but not to cover 10600–10664 MHz because that would cross the 128 MHz boundary at 10640 MHz. Note in particular that the center of a baseband is a boundary and no line should be observed at the baseband center.

The figure below illustrates these restrictions:

Correlator configuration figure: bandpass8jul12.png

The black curve shows the analog filter response for an 8-bit baseband covering 1024 MHz, starting at νBB0. The dashed blue vertical lines show the 128 MHz boundaries; no subband can cross those boundaries and 128 MHz subbands are thus constrained to cover a region between two of those boundaries, with no finer tuning being possible. Narrower subbands, like the 64 MHz subband shown here in red, can be shifted around arbitrarily within one of the 128 MHz slots, but cannot cross any of these boundaries. (The dotted vertical red lines show the boundaries of the 64 MHz subband, while the solid curve shows an illustrative line within the subband.)

The analog filter shape defining the baseband rolls off severely at one edge of the baseband, so the 128 MHz slot at that edge has reduced sensitivity. The baseband edge is at the lowest sky frequency in the baseband when using the upper sideband, and at the highest sky frequency in the baseband when using the lower sideband.

 

Subband Bandwidths and the Digital Filter Response

The bandwidth for each subband may be selected independently, and can be any of 128/2n MHz, for n= 0, 1, …, 12: 128, 64, 32, 16, 8, 4, 2, or 1 MHz, or 500, 250, 125, 62.5, or 31.25 kHz.

The usable portion of the subband is set by three effects. First, as discussed above, the analog filters which define the baseband are not perfect, leading to lower sensitivity in the 128 MHz near the baseband edge for the 8-bit samplers.

Second, because the digital filters are not infinitely sharp, the rejected sideband leaks in at both edges of the subband. This leads to additional (aliased) noise, with a factor ~2 increase in the noise at the subband edges, dropping to a few percent within a few percent of the subband edge. The precise filter shape and noise increase is a complex but predictable function of the subband bandwidth (sbBW) and the subband tuning.

The third effect stems from the offset frequencies used for sideband rejection in the WIDAR correlator. The local oscillators at the individual antennas are tuned to slightly different frequencies, with those offsets taken out in the correlator. This means that each antenna observes a slightly different sky frequency, and thus some baselines will not give an interesting correlation near one edge of the subband. The maximum frequency shift is currently set to 32×f0, with the fundamental f0 being set to f0 = max(25.6 kHz×sbBW/128 MHz, 100 Hz). Here sbBW is the smallest subband bandwidth within the baseband. For the wider subband bandwidths the maximum frequency shift corresponds to <1% of that bandwidth, but for narrower subbands the effect can be severe. For instance, a 31.25 kHz subband has f0 = 100 Hz, and a maximum frequency shift of 3.2 kHz—10% of the subband may be lost on some baselines.

 

Spectral Channels and Polarization Products

Each subband, without recirculation enabled, can have a different number of channels and polarization products, subject to two limitations:

  1. For the ithsubband, the number of spectral channels can be:
    • 64 nBlBP,i with full polarization products (RR,RL,LR,LL)
    • 128 nBlBP,i with dual polarization products (RR and LL)
    • 256 nBlBP,i with a single polarization product (RR or LL)
    Here nBlBP,i= 1, 2, 3, 4, 5, …, 64 is the number of Baseline Board Pairs (BlBPs) assigned to that subband.
  2. The sum over all subbands of nBlBP,i must be less than or equal to 64, the number of Baseline Board pairs in the correlator. Equivalently, the sum over all subbands of spectral channels times polarization products is limited to 64 × 256 = 16,384 (without recirculation).

Baseline Boards are the boards in the WIDAR correlator where the actual cross-multiplications are done. There are 128 Baseline Boards arranged as 64 Baseline Board pairs (BlBPs). The limitations given here correspond to the capabilities of the individual boards and the finite number of boards the correlator has. Use of more than one pair per subband (i.e., nBlBP>1) is known as Baseline Board stacking; see additional details about this below.

Limitation #1 corresponds to table 4.2.1 of the options for subband bandwidth and spectral resolution when using nBlBP Baseline Board pairs for a subband:

Table 4.2.1: Subband Bandwidth and Spectral Resolution Options (without recirculation)
Subband bandwidth &
total velocity coverage
Full polarization products
(RR, RL, LR, LL)
64nBlBP spectral channels

Channel spacing:
Dual polarization products
(RR and LL)
128nBlBP spectral channels
Channel spacing:
Single polarization product
(RR or LL)
256nBlBP spectral channels

Channel spacing:
128 MHz 38400/νGHz km/s 2000/nBlBP kHz 600/nBlBPGHz km/s 1000/nBlBP kHz 300/nBlBPGHz km/s 500/nBlBP kHz 150/nBlBPGHz km/s
64 19200 1000 / nBlBP 300 / nBlBP 500 / nBlBP 150 / nBlBP 250 / nBlBP 75 / nBlBP
32 9600 500 / nBlBP 150 / nBlBP 250 / nBlBP 75 / nBlBP 125 / nBlBP 37.5 / nBlBP
16 4800 250 / nBlBP 75 / nBlBP 125 / nBlBP 37.5 / nBlBP 62.5 / nBlBP 18.75 / nBlBP
8 2400 125 / nBlBP 37.5 / nBlBP 62.5 / nBlBP 18.75 / nBlBP 31.25 / nBlBP 9.375 / nBlBP
4 1200 62.5 / nBlBP 18.75 / nBlBP 31.25 / nBlBP 9.375 / nBlBP 15.625/nBlBP 4.687 /n BlBP
2 600 31.25 / nBlBP 9.375 / nBlBP 15.625/nBlBP 4.687 / nBlBP 7.8125 / nBlBP 2.344 / nBlBP
1 300 15.625/nBlBP 4.687 / nBlBP 7.8125 / nBlBP 2.344 / nBlBP 3.906 / nBlBP 1.172 / nBlBP
0.5 150 7.8125 / nBlBP 2.344 / nBlBP 3.906 / nBlBP 1.172 / nBlBP 1.953 / nBlBP 0.586 / nBlBP
0.25 75 3.906 / nBlBP 1.172 / nBlBP 1.953 / nBlBP 0.586 / nBlBP 0.977 / nBlBP 0.293 / nBlBP
0.125 37.5 1.953 / nBlBP 0.586 / nBlBP 0.977 / nBlBP 0.293 / nBlBP 0.488 / nBlBP 0.146 / nBlBP
0.0625 18.75 0.977 / nBlBP 0.293 / nBlBP 0.488 / nBlBP 0.146 / nBlBP 0.244 / nBlBP 0.073 / nBlBP
0.0325 9.375 0.488 / nBlBP 0.146 / nBlBP 0.244 / nBlBP 0.073 / nBlBP 0.122 / nBlBP 0.037 / nBlBP

Subband bandwidth and spectral resolution options. Note that the table entries refer to the spacing between spectral channels—that spacing is before any frequency smoothing, so these channels are not independent.

  • nBlBP is the number of Baseline Board Pairs assigned to the subband.
  • Each subband may have a different number of spectral channels and polarization products, and each may be tuned independently.
  • There can be at most 16 subbands per baseband, and nBlBP must be an integer: 1, 2, 3, 4, 5, …, 64.
  • The sum of nBlBP over all subbands must be less than or equal to 64.
  • Use of more than one BlBP for a subband may further restrict the number of subbands available in one or more of the basebands; see text for details.

Here are four examples of allowed general observing setups which use all 64 BlBPs to produce the maximum number of channels times polarization products:

Table 4.2.2: Example BlBP Setups
Baseband Subband

Pol'n
Products

Spectral
channels

nBlBP
Example 1 A0/C0 sb0 RR 16384 64
Example 2 A0/C0 sb0 RR 8192 32
A0/C0 sb1 RR, LL 1024 8
A0/C0 sb2 RR, LL 512 4
B0/D0 sb0 RR, LL 2048 16
B0/D0 sb1 RR,RL,LR,LL 256 4
Example 3 A0/C0 sb0 RR 8192 32
A0/C0 sb1 LL 1024 4
A0/C0 sb2 RR, LL 1024 8
A0/C0 sb3 RR,RL,LR,LL 1024 16
A0/C0 sb4 RR,RL,LR,LL 256 4
Example 4 A0/C0 sb0-5 RR,RL,LR,LL 64 6 x 1
A0/C0 sb6 RR, LL 3840 1 x 30
A0/C0 sb7 RR 768 1 x 3
A0/C0 sb8 RR,RL,LR,LL 192 1 x 3
B0/D0 sb0-2 RR,RL,LR,LL 64 3 x 1
B0/D0 sb3 LL 768 1 x 3
B0/D0 sb4 RR, LL 2048 1 x 16

 

Recirculation

When the subband bandwidth is less than the maximum 128 MHz, the spare clock cycles that become available in the correlator hardware can be re-purposed to compute additional lags using a single baseline board. This increases the spectral resolution within the subband, and is known as recirculation. Each factor of two reduction in subband bandwidth results in an additional factor of two maximum lags; therefore for subbands of 128 MHz / N the possible spectral resolution (in units of frequency) can be increased by a factor of N2.

Recirculation vs. Baseline Board Stacking

When faced with the choice between recirculation and Baseline Board stacking to increase the number of channels in a subband, we recommend recirculation for subbands narrower than 128 MHz; this is supported in observatory software (GOST, OPT). Using recirculation rather than stacking frees up more Baseline Boards for other user; alternatively the experiment becomes less dependent on all Baseline Board pairs being available/working at the time of observation. For subbands of 128 MHz, recirculation is not possible, and Baseline Board stacking must be utilized to increase the number of channels.

The present implementation of recirculation is that, for each halving of the subband bandwidth, the number of channels in the subband may be doubled without having to use additional correlator hardware. The maximum recirculation factor for a subband is 128/(subband bandwidth in MHz) and, of course, subject to other configuration restrictions such as data rate.

Baseline Board Stacking

As opposed to recirculation, which increases the number of channels in a subband by exploiting otherwise unused CPU resources, Baseline Board stacking adds more channels to a subband by adding correlator hardware resources, i.e., using up more Baseline Board pairs. Using Baseline Board stacking may therefore limit the number of subbands available in one or more of the basebands. Understanding how this works requires understanding some of the details of the correlator hardware. That understanding is built into the GOST, and observers may simply use that tool to find out whether their particular setup will, in fact, work. But the results can be confusing without some understanding of the hardware constraints from which they arise. These hardware constraints are complex, and most observers will not need to understand these details. The following section is for those who are attempting complex line experiments and who find the GOST or the RCT restricting the number of subbands and/or channels they can use in unexpected ways. Most observers can skip the following section.

Baseline Board Stacking and Correlator Use

First let us consider how the correlator hardware is organized. The cross-multiplications in the WIDAR correlator are spread across 64 Baseline Board pairs (BlBP), arranged into 4 quadrants of 16 BlBP each. Each baseband is connected directly to one of those quadrants. In the simplest mode, each of the 16 BlBP of a quadrant handles the correlations for one of the 16 subbands of the corresponding baseband. Four basebands and four quadrants are required to handle the full 8 GHz of bandwidth per polarization provided by the 3-bit (wideband) samplers: 8 GHz is split into four basebands of 2 GHz each, with each baseband fed into a different BlBP quadrant. Each BlBP in that quadrant handles a subband of maximum bandwidth 128 MHz, so 16 BlBP handles 16 subbands for a total of 16×128 MHz = 2048 MHz.

A single BlBP produces 256 cross-correlations per baseline for a single subband, which can be used for a single polarization product (e.g., RR or LL with 256 spectral channels), or two (RR and LL with 128 spectral channels each), or four (RR, RL, LR, and LL with 64 spectral channels each).

 

When using the 8-bit samplers, the total bandwidth is only 2 GHz per polarization, split into two basebands of 1 GHz each. The simplest continuum setup uses only two quadrants, since there are only two basebands; and only 8 subbands are required to span the 8×128 MHz = 1024 MHz of each baseband. Three-quarters of the correlator BlBP hardware remain unused.

 

The spectral line mode allows access to these extra correlator resources through Baseline Board stacking: using multiple BlBPs to process the same subband and produce more cross-correlations for that subband. This is done using crossbar switches which make the data for a single subband available to several BlBPs. Those BlBPs can then be used to produce more spectral channels for that subband, with n BlBPs producing 256×n cross-correlations per baseline. The limit on the total number of cross-correlations (16384) stems from the total number of BlBPs (64): 64×256 = 16384.

Unfortunately, completely flexible crossbar switches are expensive and could not be implemented in the VLA's correlator. This means that one cannot route a given subband to a randomly-chosen BlBP. The routings which are possible, are as follows:

  1. A subband in a baseband can be routed to any BlBP within the corresponding quadrant.
  2. Data coming into a given BlBP in one quadrant, can be routed to the corresponding BlBP in any other quadrant.

Routing option #1 means that one could use all the BlBPs within a quadrant to correlate a single subband, yielding 16×256 = 4096 cross-correlations for that subband:

 

Routing option #2 means that one could use the BlBPs in all 4 quadrants to correlate a single subband. One simple case would use 4 BlBPs to correlate each of the 16 subbands in a single baseband, yielding 4×256 = 1024 cross-correlations for each of those subbands. Note that in this case, no BlBPs are left to correlate any data from the second baseband.

Using routing option #2 does come with a subtle cost: assigning a BlBP in quadrant X to correlate a subband corresponding to quadrant Y removes that BlBP from use in the baseband corresponding to quadrant X…and therefore also removes the corresponding subband in that baseband. So, getting more channels for a subband in one baseband may prevent the use of a subband in a different baseband. To take a simple example, consider an experiment where one wishes to observe a single line in dual polarization with 512 channels (requiring 4 BlBPs), plus as much continuum bandwidth as possible. Naively, one would say there are 16 subbands in each baseband; one is used for the spectral line, so that leaves 16+15 = 31 subbands, and with the widest subband bandwidth (128 MHz) the total available continuum should be 31×128 MHz = 3968 MHz per polarization. Actually, however, there are only 15+15 subbands available, or 30×128 MHz = 3840 MHz per polarization, because the spectral line subband has eaten one BlBP corresponding to the other baseband:

If the same spectral line required twice as many channels, this will result in the loss of two subbands in both of the basebands:

In some cases one may want to use a different routing to use up subbands in one baseband in preference to another. For instance, the same spectral line setup (2048 cross-correlations for a single spectral line subband, plus as much continuum as possible) could be set up to allow 13 continuum subbands in the A0/C0 baseband, and the full 16 continuum subbands in B0/D0:

Understanding these confusing constraints can help observers set up the VLA more effectively to achieve their scientific goals. For instance, in a mixed line+continuum experiment, it works best to use the resource tools to set up the baseband tunings and subband channelization for the most important lines first, then add the desired continuum, and then see what correlator resources remain for any lines of secondary interest.

The above examples all use BlBP pair stacking in powers of 2, but this is not required. To give some idea of more complex possibilities, the following tables (4.2.3 and 4.2.4) give two examples of other possible configurations. The RCT display shows how the Baseline Boards are used to process the individual subbands. The cyan boxes (shaded when printed out in black and white) show the Baseline Boards used to process data from baseband A0/C0, while the yellow boxes show Baseline Boards used to process data from baseband B0/D0.

Table 4.2.3: Complex Configuration Example #1
BasebandSubbandPol'n productsSpectral channelsnBlBP
A0/C0 sb0 RR 10240 40
A0/C0 sb1 LL 768 3
A0/C0 sb2 RR,LL 2176 17
B0/D0 sb0 RR 256 1
B0/D0 sb1 RR,LL 384 3
RCT display: corr-cfg-fig:sro1_8bit_ac40+3+17_bd1+3

 

Table 4.2.4: Complex Configuration Example #2
BasebandSubbandPol'n productsSpectral channelsnBlBP
A0/C0 sb0 RR 4352 17
A0/C0 sb1 RR, LL 1152 9
B0/D0 sb0 RR,RL,LR,LL 192 3
B0/D0 sb1 RR, LL 4480 35
RCT display: corr-cfg-fig:sro2_8bit_ac17+9_bd3+35

 

The individual subbands can have different bandwidths, and those bandwidths may be chosen completely independently of the number of spectral channels in each subband. So, for instance, a subband with a bandwidth of 2 MHz and 1152 spectral channels would have a channel separation of 2 MHz/1152 = 1.736 kHz; but the observer could equally well choose a bandwidth of 64 MHz for that subband, leading to a channel separation of 64 MHz/1152 = 55.56 kHz.

Use of the 3-bit samplers further extends the possibilities. Here is one example:

Table 4.2.5: 3-bit Complex Configuration Example #1
BasebandSubbandPol'n productsSpectral channelsnBlBPQuadrant(s): Column(s)
A1/C1 sb0-8 RR, LL, RL, LR 9 x 64 9 x 1 Q1: 0–8
A1/C1 sb9 RR, LL 1 x 1152 1 x 9 Q1 & Q3: 9–11, 14 / Q4: 9
A1/C1 sb10 RR 1 x 1792 1 x 7 Q1 & Q3 & Q4 : 12,13 / Q2: 13
A1/C1 sb11 RR, LL 1 x 384 1 x 3 Q1 & Q2 & Q3: 15
A2/C2 sb0-11 RR, LL, RL, LR 12 x 64 12 x 1 Q2: 0–11
A2/C2 sb12 LL 1 x 768 1 x 3 Q2: 12, 14 / Q4: 14
B1/D1 sb0-3 RR, LL, RL, LR 4 x 64 4 x 1 Q3: 0–3
B1/D1 sb4 RR, LL, RL, LR 1 x 320 1 x 5 Q3: 4–8
B2/D2 sb0-6 RR, LL, RL, LR 7 x 64 7 x 1 Q4: 0–6
B2/D2 sb7 RR, LL 1 x 640 1 x 5 Q4: 7, 8, 10, 11, 15
RCT display: corr-cfg-fig:sro1_8bit_ac40+3+17_bd1+3

 

Once again, the GOST implements all of these constraints, and is generally smart enough to figure out the routing scheme that works best for your particular request.