From the Lead

Ryan Raba

I would like to start by saying thanks to everyone within the Common Astronomy Software Applications (CASA) development team and user community for welcoming me to this position and showing patience as I come up to speed on the complexity of this environment. CASA is a fascinating collection of utilities born from decades of hard work and deep scientific understanding. It is a profoundly challenging and rewarding endeavor to chart a course for something that is both science research and an operational program, a support library as well as a standalone application, used extensively by two observatories with different cultures and work processes, and included in a wide variety of external projects from around the world.

The CASA team has had a busy six months since the last newsletter. We hosted the annual CASA Users Committee gathering to discuss progress and feedback on a variety of topics relevant to the user community. We also conducted an internal CASA Development Face-to-Face meeting bringing together developers from the National Radio Astronomy Observatory (NRAO), the European Organisation for Astronomical Research in the Southern Hemisphere (ESO), and the National Astronomical Observatory of Japan (NAOJ) for the first time in several years. We are wrapping up the CASA 5.3 release with several new features in imaging and calibration and making very good progress on a High Performance Computing (HPC) multi-core parallelization mode for CASA processing. Support for the Very Large Array (VLA) Sky Survey, as well as upcoming Atacama Large Millimeter/submillimeter Array (ALMA) needs for Cycle 6, continue to drive testing, bug fixing, and an overall increase in maturity. We are pleased to announce the incorporation of Very Long Baseline Interferometry (VLBI) techniques into new experimental CASA tasks. Lastly, the looming discontinuation of Python 2 has forced us to begin planning the next major revision of CASA to include Python 3 migration in the next year.

I want to say thank you again for welcoming me in to the fold, and I look forward to the continued evolution of the CASA package.

CASA 5.3 Release

Juergen Ott & the CASA Team

The CASA 5.3 release is expected in May 2018 and has, as always, many exciting new features, performance improvements, and bug fixes. Highlights include the full implementation of the calibration library, n-sigma stopping criteria and improved auto-boxing, as well as multi-epoch ephemeris support in tclean, calibration table plotting in plotms, and a new task to derive the weights based on the noise of the data.

In addition, we have spent substantial effort on end-to-end testing of the CASA Parallelization framework (mpi4casa) and are getting closer to fully validating parallel imaging to the users.

Very Long Baseline Interferometry (VLBI) data reduction paths have also been demonstrated in CASA, and our colleagues from the Netherlands have contributed two new CASA tasks: fringefit and accor. Their efforts are featured in a dedicated article in this newsletter.

Our documentation effort is ongoing, and with the release of CASA 5.3, all tasks will be described in CASAdocs.

Below is a list of the improvements that the CASA team made to CASA 5.3.  After its release in May, a full list can be found in the CASA 5.3 Release Notes in CASAdocs.

Calibration

• The Calibration library is now available for the full, standard calibration paths; the calibration library enables single calls for applycal for all fields, on-the-fly application in plotms
• Implementation of Multi-band cross-hand delays
• New Perley-Butler 2017 flux standard

Imaging

• Phase centers can now be moving targets (this handles the deconvolution of multiple epoch ephemeris observations)
• New N-sigma deconvolution stopping criterion
• tclean mask extension along Stokes axis
• Improvements in memory usage for tclean in single and parallel mode
• Autoboxing performance improvements
• A new stokes parameter 'pseudoI' has been added to allow imaging data with visibilities that are partially flagged in one polarization hand but not the other.

Measurement Set (MS) visualization

• Plotms now supports plotting of virtually all calibration tables
• Calibration table summaries in plotms
• Ability to overlay atmospheric transmission curves or temperature curves to data plotted in plotms
• Header plotting in plotms
• New plotants task for better readability, including logarithmic position scaling

Analysis

• New implementation of statwt (statwt2) to work with WEIGHT/SIGMA_SPECTRUM columns, options to combine scans, fields, state, better control of data flagging, display of mean/variance, calculation of weights based on residuals, using of robust statistics
• Writing of image histories by the image analysis tasks

Single Dish

• A new task, sdsidebandsplit, was implemented to separate single-dish data observed by DSB receivers.
• A new experimental task, tsdimaging, has been defined, which is equivalent to sdimaging, but based on new imager framework.
• New interpolation option in sdcal
• Performance improvements to TP calibration, esp. for solar data

VLBI

• New tasks fringefit and accor for fringe fitting and DIFX amplitude corrections

Documentation

• All CASA tasks have now CASAdocs pages

CASA Users Committee Report

Ryan Raba

The CASA Users Committee (CUC) provides advice and feedback on the perceptions of CASA from the external user community. The committee assembles once per year to discuss progress on CASA action items and to reflect on changes and guidance from the community. Annual CUC meetings and reports are accessible here.

Current Committee members (regions):

• Adam Avison (EU): ALMA Support Scientist / Postdoctral Research Associate at the UK ALMA Regional Centre Node (Jodrell Bank Centre for Astrophysics, University of Manchester)
• Michael Bietenholz (NA): Hartebeesthoek Radio Astronomy Observatory, South Africa and York University, Canada
• Ilse van Bemmel (EU): Staff member at the Joint Institute for VLBI ERIC (JIVE)
• Chris DePree (NA): Professor of Astronomy and Director of Bradley Observatory at Agnes Scott College.
• Yi-Jehng Kuan (NA): Professor, Department of Earth Sciences, National Taiwan Normal University, Taipei, Taiwan
• Se-Heon Oh (EA): ALMA Regional Center, Science Staff, Korea Astronomy and Space Science Institute
• Shigehisa Takakuwa (EA): Professor, Kagoshima University, Japan
• Cornelia Lang (NA): Associate Professor, University of Iowa.
• D.J. Pisano (NA): Associate Professor, West Virginia University

2017 Highlights
The last CUC meeting was held 8 – 9 November 2017. Of the many topics discussed, several overarching themes emerged for the CASA team to work on. Here we elaborate on two key points that the CASA team has been working to address:

• Improved interaction with the user community including CASA promotion, feedback, surveys, help desk ticket analysis, and communication of upcoming changes.
• CASA Performance perception versus reality, gathering real benchmarks, reliability data, and usage statistics in more detail and communicating that information to the user community per CASA release.

The CUC report in its entirety can be seen here.

Improved Community Interaction
A regular challenge with any software package is how to engage the user community and elicit meaningful feedback in a way that is accurate and representative of the whole community, but also non-intrusive. Furthermore, when important software changes do arrive, they must be communicated and promoted to relevant users in a convenient and productive way.

The CASA team is studying new ways of distributing surveys, if and how to revamp the forum, and other more modern methods of promotion and collaboration such as social media. However, industry trends are shifting away from relying on active user participation and instead adopting a more passive approach by utilizing data collection and analytics. The CASA team is studying ways of assessing help desk tickets for overall trends and understanding of what users experience by way of the questions and problems that are fielded. Furthermore, a previous CUC request for better reliability data is being expanded in to a significantly more powerful capability to gather and assess real world CASA usage statistics and performance. That endeavor is intertwined with the CASA Performance assessment below.

CASA Performance
The CUC has expressed the need to better communicate hard performance expectations and reliability data for CASA to bridge the gap with user perceptions and anecdotal experience. To that end, the CASA team rolled out the Crash Reporter last year with CASA 5. The crash reporter has provided an insightful look in to what CASA tasks are unexpectedly terminating in real world usage and collected data has even been used to diagnose the problem(s). However, this is only a small part of a much larger need. Since the current Crash Reporter only collects failures, we do not see total usage and thus cannot compute a failure rate.

The CASA team is studying the myriad of related issues regarding user behavior, feedback, performance, and reliability, as well as overall industry trends in solving these issues. We are currently working to expand the crash reporter in to a more comprehensive performance collection capability that we hope will satisfy many of these outstanding challenges. We expect to debut additional functionality in the CASA 5.4 release later this year and will provide further details in the next newsletter.

User Survey and Help Desk

User Survey
The CASA team is dedicated to making CASA as easy and versatile as possible for users. In our efforts to receive feedback from the community, we compiled a short (8-question) User Survey.

This short survey is designed to give the team better insight into the wishes from users regarding:

• CASA development priorities (capabilities, speed, reliability or documentation)
• Ranking of functionalities (calibration, imaging, visualization, pipeline, etc.)
• Platforms to support in the future (currently Mac OSX and Linux/RedHat)

In addition, this survey will allow us to gain an overview for which telescopes users rely on CASA, and which geographical regions we serve. It also offers users the possibility to leave valuable feedback.

We are grateful to anyone who can take about 3 minutes time to fill out this survey-questionnaire.

NRAO and ALMA Help Desks
Another valuable resource for both users and the CASA team are the NRAO and ALMA Help Desks. Over the past seven months, 200+ CASA-related tickets were processed by the NRAO Help Desk. The combined regional ALMA Help Desks resolved a similar amount of user-reported tickets related to CASA.

We would like remind users that important information that goes through the Help Desks is captured in a convenient Knowledgebase. This Knowledgebase is a collection of articles written by staff members based on known issues that we want to convey to users, as well as frequently asked Help Desk questions. The Knowledgebase of the NRAO Help Desk also displays articles from the ALMA Help Desk knowledgebase, so that all the information can be found in one location.

To access the NRAO and ALMA Help Desks, please visit   https://help.nrao.edu/  or https://help.almascience.org/    and let us help you!

CASA on the Fringe: Developing Tools for VLBI data processing

Ilse van Bemmel (JIVE), Des Small (JIVE), Mark Kettenis (JIVE), George Moellenbrock (NRAO), Arpad Szomoru (JIVE)

CASA has become the package of choice to process and calibrate the majority of radio data, and all early career radio astronomers are now trained to use this package. However, for Very Long Baseline Interferometry (VLBI) data observers continue to use the Astronomical Image Processing System (AIPS), as CASA has so far lacked a fringe fitting task, and several other tasks have not supported the VLBI (meta-)data adequately. Fringe fitting is a crucial step in VLBI data processing, but not commonly used in connected element arrays. With support from the BlackHoleCam and Square Kilometre Array – Netherlands (SKA-NL) projects, the Joint Institute for VLBI ERIC (JIVE) has developed the new CASA tasks fringefit and accor, and upgraded other tasks to work properly with VLBI observations.

The fringefit task implements the same Schwab-Cotton global fringe fitting algorithm (Schwab & Cotton 1983) as the AIPS task FRING. Development started with building a Python proto-type, which was extensively tested against FRING. The performance was excellent, and results from the prototype and FRING were indistinguishable. The production code is fully integrated in the CASA framework, such that all methods for data selection and a priori calibration are available to the user. The figures below show the results of phase and delay corrections on a European VLBI Network (EVN) scan of a bright calibrator source: left is before, right is after correction. Non-linear bandpass phase residuals remain evident, and may be calibrated using the standard CASA bandpass task.

The new accor task implements the equivalent of the AIPS ACCOR task and re-normalizes visibilities by their auto-correlation amplitudes. Significant improvements have been made to the importfitsidi task. It now imports almost all meta-data that can be represented in a Measurement Set, and applies the same digital corrections to data correlated with DiFX software as the AIPS FITLD task. Additional improvements have been made to the tasks related to amplitude calibration using the system temperature and gain curves. Python scripts are available to append the necessary meta-data to the FITS-IDI files before importing them into CASA as a Measurement Set, and to produce CASA readable files that can be converted to calibration tables with the CASA gencal task.

After extensive in-house testing, the toolkit was at a level to be tested by experts in October 2017. During a RadioNet sponsored workshop, ~20 people were invited to test the tools on observations from a range of VLBI arrays – EVN, VLBA, Long Baseline Array, Korean VLBI Network, Low-Frequency Array, e-MERLIN, Event Horizon Telescope, and even RadioAstron. The workshop uncovered many issues that were fixed on the fly, or in later development. The code is now mature enough for an experimental release, and is included with CASA 5.3. Only basic, essential functionality will be included; the task will not yet match the full functionality of AIPS FRING. A CASA Guide is currently under development.

Several successes have already been achieved. The first VLBI dataset to be fully calibrated and imaged in CASA by Cristiana Spingola (Kapteyn Institute, Groningen) is shown in the Featured Image elsewhere in this Newsletter. These are VLBA observations of a gravitational lens system, with four lensed images and the lensing galaxy clearly visible. Since then, EVN data has also been processed and imaged successfully, and e-MERLIN, LOFAR and EHT are exploring the new CASA tasks for their data processing.

Development of the CASA VLBI tools is continuing under the RadioNet RINGS project.  Requests for implementing more functionality from AIPS FRING have already come from several astronomers, and the majority of these will be included in the full, public release of the toolkit, which is planned for this fall. The longer-term development will include fringe fitting of broad and/or non-contiguous frequency bands, and dispersive delays. For users who like a challenge, there is a dedicated mailing list where new development versions are announced.

Feedback from users on reducing VLBI data in CASA is most welcome. Please send general feedback on VLBI reduction in CASA to casa-feedback@nrao.edu, and contact the NRAO Help Desk regarding data-reduction issues or bugs.

References
Schwab, F.R., Cotton, W.D., 1983, Astronomical Journal, 88, 68.

Featured Image: VLBA observations of a lensed blazar meet the new CASA VLBI tools

Cristiana Spingola (Kapteyn Institute), Ilse van Bemmel (JIVE) and John McKean (ASTRON, Kapteyn Institute)

Very Long Baseline Interferometry (VLBI) uses multiple radio telescopes spread over large distances to achieve micro-arcsecond resolution imaging of astronomical radio sources. Traditionally, VLBI data processing has been done mostly with the AIPS software. However, as described elsewhere in this Newsletter, with the recent development of new CASA tools for VLBI data calibration, CASA has the potential to become the new software of choice for processing VLBI observations. This is an important step in lowering the threshold for non-experts to use VLBI in their astronomical research.

Shown in the image is the first VLBI science data to be fully calibrated and imaged in CASA by Cristiana Spingola (Kapteyn Institute, Groningen). The result was obtained in a dedicated workshop in late 2017 at JIVE. The observations are made with the Very Long Baseline Array (VLBA) at L-band during a 12-hour run. The target source is a gravitational lens, which is imaged here at milli-arcsecond resolution, revealing radio emission from four images of the lensed source (A through D), as well as the lensing galaxy itself. In addition to demonstrating the potential of CASA, this highlights the unique power of VLBI in resolving these lensed systems, which will enable studies of the dark matter distribution in the lensing galaxy halo in intricate detail.

The data were amplitude calibrated, phase calibrated, imaged and cleaned, all using the new and upgraded CASA VLBI tools. The new fringefit task, described elsewhere in this Newsletter, was used to determine phase and delay corrections derived from a bright calibrator source. Another brand-new CASA task for VLBI reduction is accor, which re-normalizes visibilities by their auto-correlation amplitudes. Since the production of this image, data of the European VLBI Network (EVN) have also been processed and imaged successfully. Other VLBI facilities (e.g. e-MERLIN, LOFAR and EHT) are currently testing the new CASA tasks for their data processing.

CASA Guides

Bjorn Emonts, Rachel Friesen, Emmanuel Momjian

The CASA Guides are data reduction tutorials that provide users with recipes for typical data reduction paths, based on a large variety of data from different telescopes. These Guides provide step-by-step instructions from the telescope teams on how to reduce data in CASA. The examples provided in the CASA Guides complement the official CASA documentation, which is captured in the CASA Docs.

Currently, we offer CASA Guides on the following topics:

• Karl G. Jansky Very Large Array (VLA): high-frequency spectral-line reduction, centimeter polarimetry and continuum imaging, low-frequency P-band continuum and spectral-line work. Also included are topical guides on flagging, imaging, data combination and spectral-index corrections for bandpass calibration.
• Atacama Large Millimeter Array (ALMA): a large number of ALMA Science Verification data-sets, from mosaicing and polarization observations to data from the long-baseline campaign and solar observing. Also included is a set of ‘first-look’ guides on imaging, self-calibration, spectral-line work and image analysis.
• Australia Telescope Compact Array (ATCA): advanced tutorials on continuum polarization.
• Simulations: tutorials on how to create simulated data from either existing images or component models, as well as a detailed overview of simobserve and simanalyze, which are the main CASA tasks for simulating data from various telescopes. The CASA 5.3 package includes configuration files for all current and past array-configurations of the VLA and ALMA (including the Atacama Compact Array), as well as the Combined Array for Research in Millimeter Astronomy (CARMA), Institut de Radioastronomie Millimétrique–Plateau de Bure Interferometer (IRAM-PdBI), Submillimeter Array (SMA), Westerbork Synthesis Radio Telescope (WSRT), and Next-Generation VLA (ngVLA).
• Very Long Baseline Interferometry (VLBI): the CASA Guides also include two extensive AIPS Guides for the reduction of data from the Very Long Baseline Array (VLBA). As mentioned elsewhere in this Newsletter, rapid progress is being made on the use of CASA for VLBI reduction and imaging, and a VLBI CASA Guide is currently under development.

The telescope teams continuously upgrade the CASA Guides, so that users can follow them with recent versions of CASA. All the VLA Guides were updated to CASA 5.0 in the past year. The CASA 5.0 Guides are also compatible with CASA 5.1. The VLA Guides will be updated to CASA 5.3 during the coming months. Meanwhile, a major overhaul was carried out on the ALMA and the Simulations Guides, to make them compatible with CASA 5.1. This version of CASA is currently being used in the calibration and imaging pipeline of ALMA Cycle 5 data. Both the VLA and ALMA Guides will continue to be updated in the future as new versions of CASA are released.

We hope that the CASA Guides continue to be an efficient tool to help you reduce your interferometry data and get the best science out!

Python 3: Important Changes for CASA 6

The first release of CASA was in the fall of 2009 using Python 2.5.2. In the intervening years, CASA has upgraded Python a number of times, through 2.6 versions and 2.7 versions. Currently, CASA uses Python 2.7.12. In 2020, the Python community will no longer support Python 2. Of course, this does not mean that the deployed version will suddenly cease to function, but it does mean that soon it will be time for CASA to make the move to Python 3.

Given that Python 2.7 is reaching its end of life, we thus need to formulate a plan for how to transition CASA to Python 3. Since its first release, CASA has been distributed as a single, integrated application which included a Python interpreter and all of the libraries, packages and modules that were required.

For some users, this is ideal, but other users found that a monolithic distribution makes it difficult to use CASA tools and tasks along with the other python packages for astronomy that they are accustomed to using. Their best option has been to use CASA to build a version of the other packages. Our current thinking is that, as part of the conversion to support Python 3, CASA will be reorganized to support building CASA tools and tasks with Python’s distutils (and GNU autoconf). This will allow greater flexibility for users, by building these tools for use with the version of Python that they choose to use. In addition, CASA plans to continue to provide an all-inclusive distribution under Python 3, to keep changes to a minimum for users who are not interested in these new functionalities. We are currently soliciting input from all stakeholders on these proposed changes. Users can send any feedback they have on this to casa-feedback@nrao.edu

In addition to making CASA more Pythonic in the way that it is accessible within Python, there will also be some more cleanup of problems that have accumulated over the last decade of development with Python. All of this means that a period of change is ahead for both CASA and CASA's users. To ease the transition, CASA will support two versions for one year. One version will be a single monolithic application based on Python 2.7. The other version, CASA 6.0, will be based upon Python 3. As explained above, it can be built and accessed much like other python packages in general use, but it will also be available as a complete, integrated package. The planned date for the release of CASA 6.0 is May, 2019. At that point, the clock will begin ticking for Python 2 support, and in May 2020, CASA will no longer support Python 2.

Python 2 vs Python 3
Python 3.0 was released on 3 December 2008, and it was not designed to be backward compatible. While backward compatibility was not maintained, many Python 3.0 features were backported to the later Python 2.7 versions to make it easier for users to migrate. The majority of incompatibilities that most users will face have obvious fixes, but the required fixes may sometimes be numerous. For example, all print statements must use parenthesis. So for example: print "hello world!" in Python 2.7.12, becomes: print("hello world!")

It is a simple fix, but there may be many places to fix!

Another change with Python 3 is that the standard division operator performs floating point division when the dividend and divisor are integers, whereas for Python 2 integer division was performed. So, for example, with Python 2, 5 / 2 returns 2 (an integer), but with Python 3, the result is 2.5. Both Python 2 and Python 3 do have an integer division operator. With both versions, 5 // 2 returns 2. There are many changes but most affect things like calls to C routines (e.g. with Python 3 conversion from/to unicode are required) or accessing the internal state of Python objects (e.g. what was func.func_code is now func.__code__). A good place to start to learn about the transition is this website from the Python Foundation, but there are many places to look on the internet for more information.

CASA Calendar

Dates	Event	Location
14-16 May 2018	NRAO Users Committee meeting	Socorro, NM
16-23 May 2018	16th Synthesis Imaging Workshop	Socorro, NM
4-5 June 2018	NRAO/LBO Community Day at University of Toronto	Toronto, Canada
26-29 June 2018	Conference “Astrophysical Frontiers in the Next Decade and Beyond”	Portland, OR
1 Aug 2018	NRAO Semester 2019A Proposal Deadline
4-6 Sept 2018	Young European Radio Astronomers Conference (YERAC)	JIVE/ASTRON, Dwingeloo (NL)
Oct 2018	Start of ALMA Cycle 6 observing