WIDAR Correlator: Spectral Line Observing (COPY)

Basebands and Subbands

Observers have access to very flexible correlator configurations using up to 64 subbands in up to 4 basebands sampled with the 8-bit and/or the 3-bit samplers. These capabilities may be summarized as follows:

• Two 1 GHz baseband pairs using the 8-bit samplers, or four 2 GHz baseband pairs using the 3-bit samplers, independently tunable within the limits outlined in the section on VLA Frequency Bands and Tunability. The 8-bit baseband pairs are referred to as A0/C0 and B0/D0, while the 3-bit samplers are A1/C1, A2/C2, B1/D1, and B2/D2. The AC/BD nomenclature corresponds to that of the IF pairs in the pre-expansion VLA.
• Up to 16 subband pairs (spectral windows) in each 3-bit baseband pair, and up to 32 subbands in each 8-bit baseband pair, for a total of up to 64 subbands in any correlator configuration:
  
  • Tuning, bandwidth, number of polarization products, and number of channels can be selected independently for each subband;
  • All subbands must share the same integration time;
  • No part of a subband can cross a 128 MHz boundary;
  • Subband bandwidths can be 128, 64, 32, …, 0.03125 MHz (128 / 2ⁿ, n=0, 1, …, 12).
• The sum over subbands of channels times polarization products is limited to 16384 (without recirculation):
  • These may be spread flexibly over subbands and polarization products, in multiples of 64: 64, 128, 192, 256, 384, …, 16384 cross-correlation products;
  • Recirculation may be used to increase the number of channels per subband for subbands narrower than 128 MHz. Baseline Board stacking may be used to increase the number of channels per subband for setups requiring less than 64 subbands;
  • Assigning many channels to a given subband may reduce the total bandwidth and/or the total number of subbands available.

The remainder of this section discusses the various limitations in more detail, including some examples to show how they come up in practice.

 

Subband Tuning Restrictions

Each subband may be placed anywhere within a baseband, with the caveat that no subband may cross a 128 MHz boundary. Mathematically:

νBB0 + n×128 MHz <= νsbLow <= νsbHigh <= νBB0 + (n+1)×128 MHz

where:

	νBB0	the lower frequency edge of the baseband;
	n= 0, 1, …, 7 (, …, 15)	(any integer between 0 and 7 for 8-bit, between 0 and 15 for 3-bit);
	νsbLow	the lower edge of the subband (the subband center frequency minus half the subband bandwidth);
	νsbHigh	the upper edge of the subband (the subband center frequency plus half the subband bandwidth).

For example, if the baseband were tuned to cover 10000–11024 MHz, one could place a 64 MHz subband to cover 10570–10634 MHz, but not to cover 10600–10664 MHz because that would cross the 128 MHz boundary at 10640 MHz. Note in particular that the center of a baseband is a boundary and no line should be observed at the baseband center.

The figure below illustrates these restrictions:

The black curve shows the analog filter response for an 8-bit baseband covering 1024 MHz, starting at ν_BB0. The dashed blue vertical lines show the 128 MHz boundaries; no subband can cross those boundaries and 128 MHz subbands are thus constrained to cover a region between two of those boundaries, with no finer tuning being possible. Narrower subbands, like the 64 MHz subband shown here in red, can be shifted around arbitrarily within one of the 128 MHz slots, but cannot cross any of these boundaries. (The dotted vertical red lines show the boundaries of the 64 MHz subband, while the solid curve shows an illustrative line within the subband.)

The analog filter shape defining the baseband rolls off severely at one edge of the baseband, so the 128 MHz slot at that edge has reduced sensitivity. The baseband edge is at the lowest sky frequency in the baseband when using the upper sideband, and at the highest sky frequency in the baseband when using the lower sideband.

 

Subband Bandwidths and the Digital Filter Response

The bandwidth for each subband may be selected independently, and can be any of 128/2ⁿ MHz, for n= 0, 1, …, 12: 128, 64, 32, 16, 8, 4, 2, or 1 MHz, or 500, 250, 125, 62.5, or 31.25 kHz.

The usable portion of the subband is set by three effects. First, as discussed above, the analog filters which define the baseband are not perfect, leading to lower sensitivity in the 128 MHz near the baseband edge for the 8-bit samplers.

Second, because the digital filters are not infinitely sharp, the rejected sideband leaks in at both edges of the subband. This leads to additional (aliased) noise, with a factor ~2 increase in the noise at the subband edges, dropping to a few percent within a few percent of the subband edge. The precise filter shape and noise increase is a complex but predictable function of the subband bandwidth (sbBW) and the subband tuning.

The third effect stems from the offset frequencies used for sideband rejection in the WIDAR correlator. The local oscillators at the individual antennas are tuned to slightly different frequencies, with those offsets taken out in the correlator. This means that each antenna observes a slightly different sky frequency, and thus some baselines will not give an interesting correlation near one edge of the subband. The maximum frequency shift is currently set to 32×f0, with the fundamental f0 being set to f0 = max(25.6 kHz×sbBW/128 MHz, 100 Hz). Here sbBW is the smallest subband bandwidth within the baseband. For the wider subband bandwidths the maximum frequency shift corresponds to <1% of that bandwidth, but for narrower subbands the effect can be severe. For instance, a 31.25 kHz subband has f0 = 100 Hz, and a maximum frequency shift of 3.2 kHz—10% of the subband may be lost on some baselines.

 

Spectral Channels and Polarization Products

Each subband, without recirculation enabled, can have a different number of channels and polarization products, subject to two limitations:

1. For the i^thsubband, the number of spectral channels can be:
   • 64 n_BlBP,i with full polarization products (RR,RL,LR,LL)
   • 128 n_BlBP,i with dual polarization products (RR and LL)
   • 256 n_BlBP,i with a single polarization product (RR or LL)
   Here n_BlBP,i= 1, 2, 3, 4, 5, …, 64 is the number of Baseline Board Pairs (BlBPs) assigned to that subband.
2. The sum over all subbands of n_BlBP,i must be less than or equal to 64, the number of Baseline Board pairs in the correlator. Equivalently, the sum over all subbands of spectral channels times polarization products is limited to 64 × 256 = 16,384 (without recirculation).

Baseline Boards are the boards in the WIDAR correlator where the actual cross-multiplications are done. There are 128 Baseline Boards arranged as 64 Baseline Board pairs (BlBPs). The limitations given here correspond to the capabilities of the individual boards and the finite number of boards the correlator has.

Limitation #1 corresponds to table 5.1.1 of the options for subband bandwidth and spectral resolution when using n_BlBP Baseline Board pairs for a subband:

Here are four examples of allowed general observing setups which use all 64 BlBPs to produce the maximum number of channels times polarization products:

 

Recirculation

Recirculation is a term to describe the method of increasing the number of spectral channels in a subband using correlator software, as opposed to Baseline Board stacking, which uses correlator hardware (see below). Recirculation is presently achieved by limiting the subband bandwidth and is available only for subbands less than 128 MHz wide. When limiting the bandwidth in a subband, the correlator software can be directed to use the remaining CPU cycles on a Baseline Board pair to obtain more lags in factors of two, running the data through the board for a second, third, etc., time; hence recirculation.

At some time in the future an alternative method of recirculation, using the extra CPU cycles freed up by increasing the integration time, will be made available. Recirculation, by limiting of the subband bandwidth to increase the number of channels in factors of two, was used in the pre-expansion VLA correlator.

 

Recirculation vs. Baseline Board Stacking

When faced with the choice between recirculation and Baseline Board stacking to increase the number of channels in a subband, we recommend recirculation for subbands narrower than 128 MHz and is supported in observatory software (GOST, OPT). For subbands of 128 MHz, Baseline Board stacking should be utilized to increase the number of channels.

The present implementation of recirculation is that, for each halving of the subband bandwidth, the number of channels in the subband may be doubled without having to trade off the use of other subbands. Because recirculation is achieved by limiting the subband bandwidth, it is not supported for 128 MHz subbands; therefore for 64 MHz subbands a factor 2 recirculation is supported, etc. The maximum recirculation factor for a subband is 128/(subband bandwidth in MHz) and, of course, subject to other configuration restrictions such as data rate.

The juggling between the requested number of channels, subband bandwidth, and the available number of Baseline Board pairs is dependent on the science goals and not easily formulated in a standard answer. However, if subbands of less than 128 MHz are used, recirculation becomes an option for setups that can also be achieved with Baseline Board stacking. In such cases, we suggest to use recirculation where possible and within the General Observing or Shared Risk requirements. This frees up unused Baseline Board pairs for other use; alternatively, one becomes less dependent on all Baseline Board pairs being in working order.

Recirculation with factors 8 to 64 is designated Shared Risk, and recirculation with factors over 64 as Resident Shared Risk. The latter choice may have severe implications for the sensitivity as visibility integration time is used as trade-off. Ask the NRAO Helpdesk for more details.

 

Baseline Board Stacking

As opposed to recirculation, which increases the number of channels in a subband by exploiting otherwise unused CPU resources, Baseline Board stacking adds more channels to a subband by adding correlator hardware resources, i.e., using up more Baseline Board pairs. Using Baseline Board stacking may therefore limit the number of subbands available in one or more of the basebands. Understanding how this works requires understanding some of the details of the correlator hardware. That understanding is built into the GOST, and observers may simply use that tool to find out whether their particular setup will, in fact, work. But the results can be confusing without some understanding of the hardware constraints from which they arise. These hardware constraints are complex, and most observers will not need to understand these details. The following section is for those who are attempting complex line experiments and who find the GOST or the RCT restricting the number of subbands and/or channels they can use in unexpected ways. Most observers can skip the following section.

Baseline Board Stacking and Correlator Use

First let us consider how the correlator hardware is organized. The cross-multiplications in the WIDAR correlator are spread across 64 Baseline Board pairs (BlBP), arranged into 4 quadrants of 16 BlBP each. Each baseband is connected directly to one of those quadrants. In the simplest mode, each of the 16 BlBP of a quadrant handles the correlations for one of the 16 subbands of the corresponding baseband. Four basebands and four quadrants are required to handle the full 8 GHz of bandwidth per polarization provided by the 3-bit (wideband) samplers: 8 GHz is split into four basebands of 2 GHz each, with each baseband fed into a different BlBP quadrant. Each BlBP in that quadrant handles a subband of maximum bandwidth 128 MHz, so 16 BlBP handles 16 subbands for a total of 16×128 MHz = 2048 MHz.

A single BlBP produces 256 cross-correlations per baseline for a single subband, which can be used for a single polarization product (e.g., RR or LL with 256 spectral channels), or two (RR and LL with 128 spectral channels each), or four (RR, RL, LR, and LL with 64 spectral channels each).

Baseline Board pair (BlBP) usage for a simple 4 x 2 GHz (3-bit sampler) experiment. The different colors correspond to different baseband pairs (Q1=cyan, Q2=purple, Q3=yellow, Q4=red). The BlBP quadrants are indicated along the left-hand side, while the numbers across the top represent the BlBPs within those quadrants. Numerals within the colored boxes label the subbands within the basebands. In this setup each subband is correlated using a single BlBP, using all of the 64 available BlBPs.

 

When using the 8-bit samplers, the total bandwidth is only 2 GHz per polarization, split into two basebands of 1 GHz each. The simplest continuum setup uses only two quadrants, since there are only two basebands; and only 8 subbands are required to span the 8×128 MHz = 1024 MHz of each baseband. Three-quarters of the correlator BlBP hardware remain unused.

Baseline Board pair (BlBP) usage for a simple 2 x 1 GHz (8-bit sampler) experiment. The cyan boxes (shaded when printed out in black and white) correspond to the first baseband pair (IF A0/C0), fed directly into Q1; the yellow boxes to the second (IF B0/D0), fed directly into Q3. The BlBP quadrants are indicated along the left-hand side, while the numbers across the top represent the BlBPs within those quadrants. Numerals within the colored boxes label the subbands within the basebands. In this setup each subband is correlated using a single BlBP, using only 16 of the 64 available BlBPs.

 

The spectral line mode allows access to these extra correlator resources through Baseline Board stacking: using multiple BlBPs to process the same subband and produce more cross-correlations for that subband. This is done using crossbar switches which make the data for a single subband available to several BlBPs. Those BlBPs can then be used to produce more spectral channels for that subband, with n BlBPs producing 256×n cross-correlations per baseline. The limit on the total number of cross-correlations (16384) stems from the total number of BlBPs (64): 64×256 = 16384.

Unfortunately, completely flexible crossbar switches are expensive and could not be implemented in the VLA's correlator. This means that one cannot route a given subband to a randomly-chosen BlBP. The routings which are possible, are as follows:

1. A subband in a baseband can be routed to any BlBP within the corresponding quadrant.
2. Data coming into a given BlBP in one quadrant, can be routed to the corresponding BlBP in any other quadrant.

Routing option #1 means that one could use all the BlBPs within a quadrant to correlate a single subband, yielding 16×256 = 4096 cross-correlations for that subband:

Baseline Board pair (BlBP) usage when assigning all BlBPs within a quadrant to a single subband within the corresponding baseband. The cyan boxes (shaded when printed out in black and white) correspond to the first baseband pair (IF A0/C0), fed directly into Q1; the yellow boxes to the second (IF B0/D0), fed directly into Q3. The BlBP quadrants are indicated along the left-hand side, while the numbers across the top represent the BlBPs within those quadrants. Numerals within the colored boxes label the subbands within the basebands. In this setup each subband is correlated using 16 BlBPs, yielding 16×256 = 4096 cross-correlations for those subbands, and using 32 of the 64 available BlBPs.

 

Routing option #2 means that one could use the BlBPs in all 4 quadrants to correlate a single subband. One simple case would use 4 BlBPs to correlate each of the 16 subbands in a single baseband, yielding 4×256 = 1024 cross-correlations for each of those subbands. Note that in this case, no BlBPs are left to correlate any data from the second baseband.

Baseline Board pair (BlBP) usage when assigning the BlBPs in all 4 quadrants to correlate the corresponding subbands in a single baseband. The BlBP quadrants are indicated along the left-hand side, while the numbers across the top represent the BlBPs within those quadrants. Numerals within the colored boxes label the subbands within the baseband. In this setup each subband is correlated using 4 BlBPs, yielding 4×256 = 1024 cross-correlations for those subbands, and using all of the 64 available BlBPs.

Using routing option #2 does come with a subtle cost: assigning a BlBP in quadrant X to correlate a subband corresponding to quadrant Y removes that BlBP from use in the baseband corresponding to quadrant X…and therefore also removes the corresponding subband in that baseband. So, getting more channels for a subband in one baseband may prevent the use of a subband in a different baseband. To take a simple example, consider an experiment where one wishes to observe a single line in dual polarization with 512 channels (requiring 4 BlBPs), plus as much continuum bandwidth as possible. Naively, one would say there are 16 subbands in each baseband; one is used for the spectral line, so that leaves 16+15 = 31 subbands, and with the widest subband bandwidth (128 MHz) the total available continuum should be 31×128 MHz = 3968 MHz per polarization. Actually, however, there are only 15+15 subbands available, or 30×128 MHz = 3840 MHz per polarization, because the spectral line subband has eaten one BlBP corresponding to the other baseband:

Baseline Board pair (BlBP) usage for a continuum plus single line experiment. The cyan boxes (shaded when printed out in black and white) correspond to the first baseband pair (IF A0/C0), fed directly into Q1; the yellow boxes to the second (IF B0/D0), fed directly into Q3. The BlBP quadrants are indicated along the left-hand side, while the numbers across the top represent the BlBPs within those quadrants. Numerals within the colored boxes label the subbands within the basebands. In this setup subband #15 of A0/C0 is correlated using 4 BlBPs, yielding 4×256 = 1024 cross-correlations for that subband, while the other 15 subbands in each baseband are correlated using a single BlBP each. The use of 4 BlBPs for a subband in baseband A0/C0 has eliminated the corresponding subband of the other baseband (B0/D0). Note that only 30+4 = 34 of the 64 available BlBPs have been used—there is plenty of correlator hardware available, but the signal for the spectral line subband cannot be routed to them.

 If the same spectral line required twice as many channels, this will result in the loss of two subbands in both of the basebands:

Baseline Board pair (BlBP) usage for a continuum plus single line experiment. The cyan boxes (shaded when printed out in black and white) correspond to the first baseband pair (IF A0/C0), fed directly into Q1; the yellow boxes to the second (IF B0/D0), fed directly into Q3. The BlBP quadrants are indicated along the left-hand side, while the numbers across the top represent the BlBPs within those quadrants. Numerals within the colored boxes label the subbands within the basebands. In this setup subband #14 of A0/C0 is correlated using 8 BlBPs, yielding 8×256 = 2048 cross-correlations for that subband, while the other 14 subbands in each baseband are correlated using a single BlBP each. The use of 8 BlBPs for a subband in baseband A0/C0 has eliminated one additional subband in baseband A0/C0, and two subbands in B0/D0. Note that only 28+8 = 36 of the 64 available BlBPs have been used—there is no way to route the signals from the missing subbands to that hardware.

In some cases one may want to use a different routing to use up subbands in one baseband in preference to another. For instance, the same spectral line setup (2048 cross-correlations for a single spectral line subband, plus as much continuum as possible) could be set up to allow 13 continuum subbands in the A0/C0 baseband, and the full 16 continuum subbands in B0/D0:

Baseline Board pair (BlBP) usage for a continuum plus single line experiment. The cyan boxes (shaded when printed out in black and white) correspond to the first baseband pair (IF A0/C0), fed directly into Q1; the yellow boxes to the second (IF B0/D0), fed directly into Q3. The BlBP quadrants are indicated along the left-hand side, while the numbers across the top represent the BlBPs within those quadrants. Numerals within the colored boxes label the subbands within the basebands. In this setup subband #13 of A0/C0 is correlated using 8 BlBPs, yielding 8×256 = 2048 cross-correlations for that subband, while the other 13 subbands in A0/C0 and 16 subbands in B0/D0 are correlated using a single BlBP each. In this setup the use of 8 BlBPs for a subband in baseband A0/C0 has eliminated two additional subbands in baseband A0/C0, while retaining all 16 subbands in B0/D0. Note that only 29+8 = 37 of the 64 available BlBPs have been used. Shared risk observations can assign a number of BlBPs which is not a power of two, and hence could assign one additional BlBP (Q4-BlBP 15) to the spectral line subband.

Understanding these confusing constraints can help observers set up the VLA more effectively to achieve their scientific goals. For instance, in a mixed line+continuum experiment, it works best to use the resource tools to set up the baseband tunings and subband channelization for the most important lines first, then add the desired continuum, and then see what correlator resources remain for any lines of secondary interest.

The above examples all use BlBP pair stacking in powers of 2, but this is not required. To give some idea of more complex possibilities, the following tables (5.1.3 and 5.1.4) give two examples of other possible configurations. The RCT display shows how the Baseline Boards are used to process the individual subbands. The cyan boxes (shaded when printed out in black and white) show the Baseline Boards used to process data from baseband A0/C0, while the yellow boxes show Baseline Boards used to process data from baseband B0/D0.

 

The individual subbands can have different bandwidths, and those bandwidths may be chosen completely independently of the number of spectral channels in each subband. So, for instance, a subband with a bandwidth of 2 MHz and 1152 spectral channels would have a channel separation of 2 MHz/1152 = 1.736 kHz; but the observer could equally well choose a bandwidth of 64 MHz for that subband, leading to a channel separation of 64 MHz/1152 = 55.56 kHz.

Use of the 3-bit samplers further extends the possibilities.  Here is one example:

 

Once again, the GOST implements all of these constraints, and is generally smart enough to figure out the routing scheme that works best for your particular request.