The WIDAR Correlator

Basebands and Subbands

Currently observers have access to very flexible correlator configurations using up to 64 subbands in up to 4 basebands sampled with the 8-bit and/or the 3-bit samplers.  These capabilities may be summarized as follows:

• Two 1 GHz baseband pairs using the 8-bit samplers, or four 2 GHz baseband pairs using the 3-bit samplers, independently tunable within the limits outlined in the section on VLA Frequency Bands and Tunability. The 8-bit baseband pairs are referred to as A0/C0 and B0/D0, while the 3-bit samplers are A1/C1, A2/C2, B1/D1, and B2/D2.  The AC/BD nomenclature corresponds to that of the IF pairs in the pre-expansion VLA.
• Up to 16 subband pairs (spectral windows) in each 3-bit baseband pair, and up to 32 subbands in each 8-bit baseband pair with a total of up to 64 subbands in any correlator configuration
  
  • Tuning, bandwidth, number of polarization products, and number of channels can be selected independently for each subband
  • All subbands must share the same integration time
  • No part of a subband can cross a 128 MHz boundary
  • Subband bandwidths can be 128, 64, 32, ..., 0.03125 MHz (128 / 2ⁿ, n=0, 1, ..., 12)
• The sum over subbands of channels times polarization products is limited to 16,384 (without recirculation).
  • These may be spread flexibly over subbands and polarization products, in multiples of 64: 64, 128, 192, 256, 384, ..., 16384 cross-correlation products.
  • Recirculation may be used to increase the number of channels per subband for subbands narrower than 128 MHz, Baseline Board stacking may be used to increase the number of channels per subband for setups requiring less than 64 subbands.
  • Assigning many channels to a given subband may reduce the total bandwidth and/or the total number of subbands available.

The remainder of this section discusses the various limitations in more detail, including some examples to show how they come up in practice.

Subband tuning restrictions

Each subband may be placed anywhere within a baseband, with the caveat that no subband may cross a 128 MHz boundary. Mathematically:

where:

So for example, if the baseband were tuned to cover 10000-11024 MHz, one could place a 64 MHz subband to cover 10570-10634 MHz, but not to cover 10600-10664 MHz (because that would cross the 128 MHz boundary at 10640 MHz). Note in particular that the center of a baseband is a boundary and no line should be observed at the baseband center.

The figure below illustrates these restrictions:

The black curve shows the analog filter response for an 8-bit baseband covering 1024 MHz, starting at ν_BB0. The dashed blue vertical lines show the 128 MHz boundaries; no subband can cross those boundaries.  128 MHz subbands are thus constrained to cover a region between two of those boundaries, and no finer tuning is possible.  Narrower subbands, like the 64 MHz subband shown here in red, can be shifted around arbitrarily within one of the 128 MHz "slots", but cannot cross any of these boundaries. (The dotted vertical red lines show the boundaries of the 64 MHz subband, while the solid curve shows an illustrative line within the subband.)

The analog filter shape defining the baseband rolls off severely at one edge of the baseband, so the 128 MHz "slot" at that edge has reduced sensitivity.  The baseband edge is at the lowest sky frequency in the baseband when using upper sideband, and at the highest sky frequency in the baseband when using lower sideband.

Subband bandwidths & the digital filter response

The bandwidth for each subband may be selected independently, and can be any of 128/2ⁿ MHz, for n= 0, 1, ..., 12: 128, 64, 32, 16, 8, 4, 2, or 1 MHz, or 500, 250, 125, 62.5, or 31.25 kHz.

The usable portion of the subband is set by three effects. First, as discussed above, the analog filters which define the baseband are not perfect, leading to lower sensitivity in the 128 MHz near the baseband edge for the 8-bit samplers

Second, because the digital filters are not infinitely sharp, the rejected sideband leaks in at both edges of the subband. This leads to additional (aliased) noise, with a factor ~2 increase in the noise at the subband edges, dropping to a few per cent within a few per cent of the subband edge. The precise filter shape and noise increase is a complex but predictable function of the subband bandwidth and the subband tuning.

The third effect stems from the offset frequencies used for sideband rejection in the WIDAR correlator. The local oscillators at the individual antennas are tuned to slightly different frequencies, with those offsets taken out in the correlator. This means that each antenna observes a slightly different sky frequency, and thus some baselines will not give an interesting correlation near one edge of the subband. The maximum frequency shift is currently set to 32*f0, with the fundamental f0 being set to f0= max(25.6 kHz*sbBW/128 MHz, 100 Hz). Here sbBW is the smallest subband bandwidth within the baseband. For the wider subband bandwidths the maximum frequency shift corresponds to <1% of that bandwidth, but for narrower subbands the effect can be severe. For instance, a 31.25 kHz subband has f0= 100 Hz, and a maximum frequency shift of 3.2 kHz -- 10% of the subband may be lost on some baselines.

Spectral channels and polarization products

Each subband (without recirculation enabled) can have a different number of channels and polarization products, subject to two limitations:

1. For the i^thsubband, the number of spectral channels can be:
   • 64 n_BlBP,i with full polarization products (RR,RL,LR,LL)
   • 128 n_BlBP,i with dual polarization products (RR,LL)
   • 256 n_BlBP,i with a single polarization product (RR or LL)
   Here n_BlBP,i= 1, 2, 3, 4, 5, ..., 64 is the number of Baseline Board Pairs (BlBPs) assigned to that subband.
2. The sum over all subbands of n_BlBP,i must be less than or equal to 64, the number of Baseline Board pairs in the correlator. Equivalently, the sum over all subbands of spectral channels times polarization products is limited to 64x256= 16,384 (without recirculation).

Baseline Boards are the boards in the WIDAR correlator where the actual cross-multiplications are done. There are 128 Baseline Boards arranged as 64 Baseline Board pairs (BlBPs).  The limitations given here correspond to the capabilities of the individual boards, and the finite number of boards the correlator has.

Limitation #1 corresponds to the following table of the options for subband bandwidth and spectral resolution when using n_BlBP Baseline Board pairs for a subband:

Here are four examples of allowed general observing setups which use all 64 BlBPs to produce the maximum number of channels times polarization products:

Recirculation

Recirculation is a term to describe the method of increasing the number of spectral channels in a subband using correlator software (as opposed to Baseline Board stacking which uses correlator hardware, see below). Currently recirculation is achieved by limiting the subband bandwidth and thus only available for subbands less than 128 MHz wide.  When limiting the bandwidth in a subband, the correlator software can be directed to use the remaining CPU cycles on a Baseline Board pair to obtain more lags (in factors of two), running the data through the board for a second, third, etc., time; hence Recirculation.

At some time in the future an alternative method of recirculation, using the extra CPU cycles freed up by increasing the integration time, would be made available. Recirculation by limiting of the subband bandwidth to increase the number of channels (in factors of two) was used in the pre-expansion VLA correlator.

Recirculation versus Baseline Board stacking

When faced with the choice between Recirculation and Baseline Board stacking (see below) to increase the number of channels in a subband for subbands narrower than 128 MHz we recommend the former, which is supported in observatory software (GOST, OPT). For 128 MHz subbands Baseline Board Stacking should be utilized to increase the number of channels.

The current implementation of Recirculation is that for each halving of the subband bandwidth the number of channels in the subband may be doubled without having to trade off the use of other subbands. Because recirculation is achieved by limiting the subband bandwidth, it is not supported for 128 MHz subbands, whereas for 64 MHz subbands only a factor 2 recirculation is supported, etc. The maximum recirculation factor for a subband is 128/(subband bandwidth in MHz), and of course also subject to other configuration restrictions such as data rate.

The juggling between the requested number of channels, subband bandwidth and the available number of Baseline Board pairs is dependent on the science goals and not easily formulated in a standard answer. However, if subbands of less than 128 MHz are used, Recirculation becomes an option for setups that can also be achieved with Baseline Board stacking. In such cases we suggest to use Recirculation where possible, and within the General Observing or Shared Risk requirements.  This frees up unused Baseline Board pairs for other use; alternatively, one becomes less dependent on all Baseline Board pairs being in working order.

Recirculation with factors 8 to 64 is designated Shared Risk, and recirculation with factors over 64 Resident Shared Risk. The latter choice may have severe implications for the sensitivity as visibility integration time is used as trade-off. Ask the NRAO Helpdesk for more details.

Baseline Board stacking

As opposed to recirculation, which increases the number of channels in a subband by exploiting otherwise unused CPU resources, Baseline Board Stacking adds more channels to a subband by adding correlator hardware resources, i.e. using up more Baseline Board pairs.  Using Baseline Board stacking may therefore limit the number of subbands available in one or more of the basebands.  Understanding how this works requires understanding some of the details of the correlator hardware. That understanding is built into the GOST, and observers may simply use that tool to find out whether their particular setup will, in fact, work. But the results can be confusing without some understanding of the hardware constraints from which they arise.  These hardware constraints are complex, and most observers will not need to understand these details.  The following section is for those few who are attempting complex line experiments, and who find the GOST or the RCT restricting the number of subbands and/or channels they can use in unexpected ways  Most observers can skip it.

Baseline Board Stacking and Correlator Use

First let us consider how the correlator hardware is organized. The cross-multiplications in the WIDAR correlator are spread across 64 Baseline Board (BlB) pairs, arranged into 4 quadrants of 16 BlB pairs each. Each baseband is connected directly to one of those quadrants. In the simplest mode, each of the 16 BlB pairs of a quadrant handles the correlations for one of the 16 subbands of the corresponding baseband. Four basebands and four quadrants are required, in order to handle the full 8 GHz of bandwidth per polarization provided by the 3-bit (wideband) samplers: that 8 GHz is split into four basebands of 2 GHz each, with each baseband fed into a different BlB quadrant. Each BlB pair in that quadrant handles a subband of maximum bandwidth 128 MHz, so 16 BlB pairs handles 16 subbands for a total of 16x128 MHz= 2048 MHz.

A single BlB pair produces 256 cross-correlations per baseline for a single subband, which can be used for a single polarization product (e.g., RR with 256 spectral channels), or two (RR+LL with 128 spectral channels each), or four (RR,RL,LR,LL, with 64 spectral channels each).

Baseline Board (BlB) pair usage for a simple 4 x 2 GHz (3-bit sampler) experiment. The different colors correspond to different baseband pairs. The BlB quadrants are indicated along the left-hand side, while the numbers across the top represent the BlB pairs within those quadrants. Numerals within the colored boxes label the subbands within the basebands. In this setup each subband is correlated using a single BlB pair, using all of the 64 available BlB pairs.

When using the 8-bit samplers, the total bandwidth is only 2 GHz per polarization, split into two basebands of 1 GHz each. The simplest continuum setup uses only two quadrants, since there are only two basebands; and only 8 subbands are required to span the 8x128 MHz= 1024 MHz of each baseband. Three-quarters of the correlator BlB hardware remain unused.

Baseline Board (BlB) pair usage for a simple 2 x 1 GHz (8-bit sampler) experiment. The cyan boxes correspond to the first baseband pair (IF A0/C0), fed directly into Q1; the yellow boxes to the second (IF B0/D0), fed directly into Q3. The BlB quadrants are indicated along the left-hand side, while the numbers across the top represent the BlB pairs within those quadrants. Numerals within the colored boxes label the subbands within the basebands. In this setup each subband is correlated using a single BlB pair, using only 16 of the 64 available BlB pairs.

The spectral line mode allows access to these `extra' correlator resources through Baseline Board stacking: using multiple BlB pairs to process the same subband and produce more cross-correlations for that subband. This is done using crossbar switches which make the data for a single subband available to several BlB pairs. Those BlB pairs can then be used to produce more spectral channels for that subband, with n BlB pairs producing 256*n cross-correlations per baseline. The limit on the total number of cross-correlations (16384) stems from the total number of BlB pairs (64): 64x256= 16384.

Unfortunately completely flexible crossbar switches are expensive, and could not be implemented in the VLA's new correlator. This means that one cannot route a given subband to a randomly-chosen BlB pair. The routings which are possible, are as follows:

1. A subband in a baseband can be routed to any BlB pair within the corresponding quadrant.
2. Data coming into a given BlB pair in one quadrant, can be routed to the corresponding BlB pair in any other quadrant.

Routing option #1 means that one could use all the BlB pairs within a quadrant to correlate a single subband, yielding 16x256= 4096 cross-correlations for that subband:

Baseline Board (BlB) pair usage when assigning all BlB pairs within a quadrant to a single subband within the corresponding baseband. The cyan boxes correspond to the first baseband pair (IF A0/C0), fed directly into Q1; the yellow boxes to the second (IF B0/D0), fed directly into Q3. The BlB quadrants are indicated along the left-hand side, while the numbers across the top represent the BlB pairs within those quadrants. Numerals within the colored boxes label the subbands within the basebands. In this setup each subband is correlated using 16 BlB pairs, yielding 16x256= 4096 cross-correlations for those subbands, and using 32 of the 64 available BlB pairs.

Routing option #2 means that one could use the BlB pairs in all 4 quadrants to correlate a single subband. One simple case would use 4 BlB pairs to correlate each of the 16 subbands in a single baseband, yielding 4x256= 1024 cross-correlations for each of those subbands. Note that in this case, no BlB pairs are left to correlate any data from the second baseband.

Baseline Board (BlB) pair usage when assigning the BlB pairs in all 4 quadrants to correlate the corresponding subbands in a single baseband. The BlB quadrants are indicated along the left-hand side, while the numbers across the top represent the BlB pairs within those quadrants. Numerals within the colored boxes label the subbands within the baseband. In this setup each subband is correlated using 4 BlB pairs, yielding 4x256= 1024 cross-correlations for those subbands, and using all of the 64 available BlB pairs.

Using routing option #2 does come with a subtle cost: assigning a BlB pair in quadrant X to correlate a subband corresponding to quadrant Y, removes that BlB pair from use in the baseband corresponding to quadrant X...and hence also removes the corresponding subband in that baseband. So getting more channels for a subband in one baseband, may prevent the use of a subband in a different baseband. To take a simple example, consider an experiment where one wishes to observe a single line in dual polarization with 512 channels (requiring 4 BlB pairs), plus as much continuum bandwidth as possible. Naively one would say there are 16 subbands in each baseband; one is used for the spectral line, so that leaves 16+15= 31 subbands, and with the widest subband bandwidth (128 MHz) the total available continuum should be 31x128 MHz= 3968 MHz per polarization. Actually however there are only 15+15 subbands available, or 30x128 MHz= 3840 MHz per polarization, because the spectral line subband has "eaten" one BlB pair corresponding to the other baseband:

Baseline Board (BlB) pair usage for a continuum plus single line experiment. The cyan boxes correspond to the first baseband pair (IF A0/C0), fed directly into Q1; the yellow boxes to the second (IF B0/D0), fed directly into Q3. The BlB quadrants are indicated along the left-hand side, while the numbers across the top represent the BlB pairs within those quadrants. Numerals within the colored boxes label the subbands within the basebands. In this setup subband #15 of A0/C0 is correlated using 4 BlB pairs, yielding 4x256= 1024 cross-correlations for that subband, while the other 15 subbands in each baseband are correlated using a single BlB pair each. The use of 4 BlB pairs for a subband in baseband A0/C0 has eliminated the corresponding subband of the other baseband (B0/D0). Note that only 30+4= 34 of the 64 available BlB pairs have been used -- there is plenty of correlator hardware available, but the signal for the spectral line subband cannot be routed to them.

If the same spectral line required twice as many channels, this result in the loss of two subbands in both of the basebands:

Baseline Board (BlB) pair usage for a continuum plus single line experiment. The cyan boxes correspond to the first baseband pair (IF A0/C0), fed directly into Q1; the yellow boxes to the second (IF B0/D0), fed directly into Q3. The BlB quadrants are indicated along the left-hand side, while the numbers across the top represent the BlB pairs within those quadrants. Numerals within the colored boxes label the subbands within the basebands. In this setup subband #14 of A0/C0 is correlated using 8 BlB pairs, yielding 8x256= 2048 cross-correlations for that subband, while the other 14 subbands in each baseband are correlated using a single BlB pair each. The use of 8 BlB pairs for a subband in baseband A0/C0 has eliminated one additional subband in baseband A0/C0, and two subbands in B0/D0. Note that only 28+8= 36 of the 64 available BlB pairs have been used -- there is no way to route the signals from the "missing" subbands to that hardware.

In some cases one may want to use a different routing, to use up subbands in one baseband in preference to another. For instance, the same spectral line setup (2048 cross-correlations for a single spectral line subband, plus as much continuum as possible) could be set up to allow 13 "continuum" subbands in the A0/C0 baseband, and the full 16 "continuum" subbands in B0/D0:

Baseline Board (BlB) pair usage for a continuum plus single line experiment. The cyan boxes correspond to the first baseband pair (IF A0/C0), fed directly into Q1; the yellow boxes to the second (IF B0/D0), fed directly into Q3. The BlB quadrants are indicated along the left-hand side, while the numbers across the top represent the BlB pairs within those quadrants. Numerals within the colored boxes label the subbands within the basebands. In this setup subband #13 of A0/C0 is correlated using 8 BlB pairs, yielding 8x256= 2048 cross-correlations for that subband, while the other 13 subbands in A0/C0 and 16 subbands in B0/D0 are correlated using a single BlB pair each. In this setup the use of 8 BlB pairs for a subband in baseband A0/C0 has eliminated two additional subbands in baseband A0/C0, while retaining all 16 subbands in B0/D0. Note that only 29+8= 37 of the 64 available BlB pairs have been used. [Shared risk observations can assign a number of BlB pairs which is not a power of two, and hence could assign one additional BlBP (Q4-BlBP 15) to the spectral line subband.]

Understanding these confusing constraints can help observers set up the VLA more effectively to achieve their scientific goals. For instance, for a mixed line+continuum experiment it works best to use the resource tools to set up the baseband tunings and subband channelization for the most important lines first, then add the desired continuum, and then see what correlator resources remain for any lines of secondary interest.

The above examples all use BlB pair stacking in powers of 2, but this is not required.  To give some idea of more complex possibilities, the following tables give two examples of other possible configurations. The RCT display shows how the Baseline Boards are used to process the individual subbands. The cyan boxes show the Baseline Boards used to process data from baseband A0/C0, while the yellow boxes show Baseline Boards used to process data from baseband B0/D0.

Complex Configuration Example #1
Baseband	Subband	Pol'n products	Spectral channels	nBlBP
A0/C0	sb0	RR	10240	40
A0/C0	sb1	LL	768	3
A0/C0	sb2	RR,LL	2176	17
B0/D0	sb0	RR	256	1
B0/D0	sb1	RR,LL	384	3
RCT display:

Complex Configuration Example #2
Baseband	Subband	Pol'n products	Spectral channels	nBlBP
A0/C0	sb0	RR	4352	17
A0/C0	sb1	RR, LL	1152	9
B0/D0	sb0	RR,RL,LR,LL	192	3
B0/D0	sb1	RR, LL	4480	35
RCT display:

Note that the individual subbands can have different bandwidths, and those bandwidths may be chosen completely independently of the number of spectral channels in each subband. So for instance a subband with bandwidth 2 MHz and 1152 spectral channels would have a channel separation of 2 MHz/1152= 1.736 kHz; but the observer could equally well choose a bandwidth of 64 MHz for that subband, leading  to a channel separation of 64 MHz/1152=  55.56 kHz.

Use of the 3-bit samplers further extends the possibilities.  Here is one example:

3-bit Complex Configuration Example #1
Baseband	Subband	Pol'n products	Spectral channels	nBlBP
A1/C1	sb0-8	RR, LL, RL, LR	9 x 64	9 x 1
A1/C1	sb9	RR, LL	1 x 1152	1 x 9
A1/C1	sb10	RR	1 x 1792	1 x 7
A1/C1	sb11	RR, LL	1 x 384	1 x 3
A2/C2	sb0-11	RR, LL, RL, LR	12 x 64	12 x 1
A2/C2	sb12	LL	1 x 768	1 x 3
B1/D1	sb0-3	RR, LL, RL, LR	4 x 64	4 x 1
B1/D1	sb4	RR, LL, RL, LR	1 x 320	1 x 5
B2/D2	sb0-6	RR, LL, RL, LR	7 x 64	7 x 1
B2/D2	sb7	RR, LL	1 x 640	1 x 5
RCT display:

Once again, the GOST implements all of these constraints, and is generally smart enough to figure out the routing scheme that works best for your particular request.