Welcome to the VLASS Technical Working Group (TWG) Sub-Forum

smyers
Posts: 29
Joined: Mon Sep 09, 2013 4:43 pm

Welcome to the VLASS Technical Working Group (TWG) Sub-Forum

Postby smyers » Tue Feb 18, 2014 6:50 pm

Welcome to the Jansky VLA Sky Survey (VLASS) Technical Working Group (TWG) Forum!

This forum is for discussion of technical issues and development areas related to the survey and overall design, and other aspects related to the issues and techniques for observing, the processing of the survey data, and production of data products. Includes: scheduling, observing, calibration, imaging, object identification and catalogs, error analysis and likelihood functions, optimal array configurations, simulations, and other issues like software and algorithms research and development. See https://science.nrao.edu/science/surveys/vlass for details on the VLASS initiative.

To kick off discussion, here are some initial questions to ponder:

- What do you see as the most important decision to be made by the TWG?

- What simulations of performance should be done?

- What software technologies might be explored and utilized in processing and analysis?

- Upgrade planning! How should the processing and analysis grow and develop if the survey progresses over a number of years?

These are just examples of the issues we might address. We will also be responsive to issues that come up during the discussion in the other groups that need technical answers. This WG will also interface with the VLASS Survey Design Group (SDG) that I (Myers) lead that will have to do the detailed planning for the survey proposal and the implementation.

All users that wish to contribute to the forum discussions are required to register via http://my.nrao.edu. See the Terms of Use post for guidelines on use of this forum.

We look forward to your participation in the VLASS!

Regards,
VLASS Technical Working Group Co-Chairs: Casey Law (UC Berkeley) and Steve Myers (NRAO)

mlacy
Posts: 1
Joined: Fri Jul 22, 2011 9:48 am

Re: Welcome to the VLASS Technical Working Group (TWG) Sub-F

Postby mlacy » Tue Feb 25, 2014 9:44 pm

OK, here's my 2c:

- What do you see as the most important decision to be made by the TWG?

Hard to tell until we know the survey parameters.

- What simulations of performance should be done?

Again, it depends on the survey, looking at new arrays optimized for a range of angular scale sensitivity might be useful though. For the rest, using real data may be the way to go - there is now a lot of archival data, plus I'm sure asking around we can get some more (for example, I have a handful of pointings for a C-priority L-band survey of Lockman last semester I'd be happy to share), using this may be the best way to test processing and source measurement algorithms.

- What software technologies might be explored and utilized in processing and analysis?

The VLA pipeline seems to be doing a good job of calibration. Mapping in CASA is still very slow, but seems to be giving good results, though if rapid response to transients is needed we may need alternatives. The biggest challenge may be source finding and fitting, certainly if we want to use CASA throughout we would need to do some additional coding and upgrades to the existing software.

- Upgrade planning! How should the processing and analysis grow and develop if the survey progresses over a number of years?

Staged data releases may be a good way to go - at each stage we can reprocess the entire dataset obtained to date with updated software (this is what SDSS did).

Ian_Heywood
Posts: 3
Joined: Mon Jun 04, 2012 7:13 am

Re: Welcome to the VLASS Technical Working Group (TWG) Sub-F

Postby Ian_Heywood » Tue Mar 11, 2014 2:26 am

Maybe we should break out a few of Steve's headings into separate threads.

Here are a couple of thoughts:

- Exploration of a hybrid configuration could be done pretty readily in a 'brute force' fashion, i.e. compute the PSF characteristics of many hybrid arrays as a function of Declination (and in the case of snapshot observations, hour angle) and try to identify the resolution / sidelobe-level sweet spot. Is this something that would be useful?

- I've been using the VLA CASA pipeline extensively and it's absolutely fantastic in terms of being a labour saving device but it's still not all things to all pointings. It tends to produce data that are self-cal ready, and in a few cases observations with obvious bad data can slip past the flagging stage. I've been working on automatic self-cal schemes in CASA with some success (one of the projects I'm processing is about a thousand pointings), as well as trying to automate direction-dependent calibration using MeqTrees for problematic bright off-axis sources in the case of a separate deeper survey. I am happy to share my findings.

- Ignoring any potential politics for a moment, I would advocate using whatever software we can get our hands on that gets us a reliable result in the fastest way. The recent CALIM workshop featured some promising alternatives to the CLEAN algorithm, and some new imagers that seem to be very fast.

smyers
Posts: 29
Joined: Mon Sep 09, 2013 4:43 pm

Re: Welcome to the VLASS Technical Working Group (TWG) Sub-F

Postby smyers » Tue Mar 11, 2014 12:20 pm

Ian_Heywood wrote:Maybe we should break out a few of Steve's headings into separate threads.


Agreed - we should start threads on more detailed discussions of the various issues. For now here are some quick comments.

Ian_Heywood wrote:- Exploration of a hybrid configuration could be done pretty readily in a 'brute force' fashion, i.e. compute the PSF characteristics of many hybrid arrays as a function of Declination (and in the case of snapshot observations, hour angle) and try to identify the resolution / sidelobe-level sweet spot. Is this something that would be useful?


Its a good start. Though I think we probably need some actual Clean MFS reconstructions of simulated images with sources of different shapes and sizes and spectral indexes (for different components). One of the big new features of JVLA over VLA-Classic is the wide bandwidth and possibility of MFS imaging, and we should show what that can do. But starting with simple PSF generation is a good idea.

Ian_Heywood wrote:- I've been using the VLA CASA pipeline extensively and it's absolutely fantastic in terms of being a labour saving device but it's still not all things to all pointings. It tends to produce data that are self-cal ready, and in a few cases observations with obvious bad data can slip past the flagging stage. I've been working on automatic self-cal schemes in CASA with some success (one of the projects I'm processing is about a thousand pointings), as well as trying to automate direction-dependent calibration using MeqTrees for problematic bright off-axis sources in the case of a separate deeper survey. I am happy to share my findings.


Calibration is fairly straightforward. I use my own hacked version of the VLA CASA Pipeline which disables all the plotting which runs in about 2/3 the time. For my OTF stuff I have an even simpler pared down script that runs in about 20% of the time of the full up pipeline. One interesting thing is that the JVLA Online system does calibration in real time (how it sets initial delays etc) and that data can actually be accessed and used to calibrate your data! Casey had played around with this and can elaborate. Might be good as an initial pass to set up initial flagging.

I think Flagging will be the key thing and probably more time consuming than calibration itself.

Ian_Heywood wrote:- Ignoring any potential politics for a moment, I would advocate using whatever software we can get our hands on that gets us a reliable result in the fastest way. The recent CALIM workshop featured some promising alternatives to the CLEAN algorithm, and some new imagers that seem to be very fast.


Agreed, particularly for fast quasi real-time cal/imaging for transients. We (Gregg Hallinan's group) have been using Stephen Bourke's AIPSLite for our calibration on Stripe82 S-band with high efficiency, then passing the data to CASA for MFS imaging. There are other interesting options out there as you say. We may end up with a bit of a Frankenstein's Monster pipeline!

claw
Posts: 26
Joined: Mon Mar 03, 2014 7:18 pm

Re: Welcome to the VLASS Technical Working Group (TWG) Sub-F

Postby claw » Tue Mar 11, 2014 1:56 pm

Regarding simulations of the PSF, they are absolutely necessary and any effort in that direction would be welcome. A key question is whether JVLA's wide bandwidths will drive us to a VLASS-specific antenna configuration. If we can show that we *don't* need a custom configuration, that would be valuable too, since it would simplify logistics. Given that the antenna spacing was chosen before MFS was developed, I'm guessing that there will be benefit to a custom configuration.
The calibration products provided by the online system are remarkably good. They generally compare well with my own, manual, CASA calibration products. Some in Socorro note that they could be applied on the fly, but that makes some nervous ("do no harm"!). Alternatively, they could be applied to help flagging without permanently changing the calibration state.
This topic raises another important issue: how much new software will be needed to bring the VLASS together? I have no qualms about developing code outside of NRAO purview, but it may not be necessary. The VLASS will be a substantial NRAO investment, so they'll certainly support some code development/testing. The question is: what do we feel we need beyond what currently exists?

sbhatnag
Posts: 3
Joined: Wed Oct 05, 2011 3:39 pm

Re: Welcome to the VLASS Technical Working Group (TWG) Sub-F

Postby sbhatnag » Tue Mar 11, 2014 4:30 pm

I think one important decision that the TWG can provide useful input on is: the algorithms that the VLASS imaging will need to trigger. IMO, VLASS may not need the most advanced (often also the most expensive) algorithms.

The run-time cost varies significantly from one algorithm to another. So does the imaging performance. The required imaging dynamic range (though DR is not the only and in some cases on even the best metric to use for imaging performance) in turn will determine, to a large extend, the algorithms we need. Deep wide-band wide-field imaging will be the most demanding in terms of the cost of imaging. A shallower wide-band wide-field survey may get-by without many direction-dependent corrections (which are expensive) while still be deeper than existing surveys. Similarly for full-pol. imaging -- do we need off-axis corrections and to what level? Answer will have a significant impact on computing cost. Since it will also be mosaic imaging, the issue of memory footprint will get important -- and is also a strong function of the algorithm used.

So in my mind, the key parameter the TWG would need to know is the thermal noise limit *and* the imaging dynamic range, separately for each Stokes.

For simulations, in my work involving wide-band imaging with the EVLA and estimating the computing costs, I have found that it is most useful to do simulations to test the *actual* imaging fidelity one gets. This also gives a handle on the question posed above. Given an imaging DR requirement, one can determine which algorithms will get us there. This gives a more direct measure of the actual run-time, memory footprints, and the scientific reliability of images (which is often not reflected in the DR metric alone). I expect that such a simulation effort will also give us a good handle on the other, very important question that Steve posed: What new, if any, development may be required? What software package/technology to use?

Also, assuming we are going to have to deconvolve, properties of the wide-band PSF seem less important. Cost of deconvolution and even imaging performance is a weak function of the PSF properties and is a stronger function of the algorithm use to generate model (e.g. multi-scale vs. delta function basis vs. other basis with fancier names, wide-band modeling, full-pol modeling, etc.).

sanjay


Return to “Technical Working Group”

Who is online

Users browsing this forum: No registered users and 2 guests