supportsci:tools
Differences
This shows you the differences between two versions of the page.
Next revision | Previous revision | ||
supportsci:tools [2018/07/26 11:55] – created - list of tools jay | supportsci:tools [2019/04/08 11:58] (current) – [EEE(2)] benito | ||
---|---|---|---|
Line 1: | Line 1: | ||
====== Support Tools ====== | ====== Support Tools ====== | ||
+ | |||
+ | =====Personal Computers===== | ||
+ | **schedgaps.py** checks the ' | ||
+ | < | ||
+ | usage: schedgaps.py [-h] [-e EARLY] sum_file | ||
+ | |||
+ | List gaps for each telescope in a SCHED keyin file. Note, stations with | ||
+ | continuous cal will be reported but have an asterix in front of them | ||
+ | |||
+ | positional arguments: | ||
+ | sum_file | ||
+ | sumitem or by itself (last) | ||
+ | |||
+ | optional arguments: | ||
+ | -h, --help | ||
+ | -e EARLY, --early EARLY | ||
+ | seconds antenna must be on source before scan start to | ||
+ | count as sufficient for tsys. Defaults to 11 | ||
+ | </ | ||
=====CCSBETA===== | =====CCSBETA===== | ||
Line 5: | Line 24: | ||
< | < | ||
usage: make_lis [-h] -e EXPERIMENT [-p PROFILE] [-m MPCSOURCE] [-s SAVE] | usage: make_lis [-h] -e EXPERIMENT [-p PROFILE] [-m MPCSOURCE] [-s SAVE] | ||
- | make_lis: error: argument | + | |
+ | Create LIS files | ||
+ | |||
+ | optional arguments: | ||
+ | -h, --help | ||
+ | | ||
+ | -p PROFILE, --profile PROFILE | ||
+ | Profile to filter for. Default: use all profiles. | ||
+ | -m MPCSOURCE, --mpcsource MPCSOURCE | ||
+ | Multi phase center source to include calibrators. | ||
+ | Default: all sources. | ||
+ | -s SAVE, --save SAVE Save the LIS file to file. Default: < | ||
+ | in current directory. For multi phase center | ||
+ | experiments, | ||
+ | where ' | ||
+ | this case the default | ||
</ | </ | ||
+ | |||
+ | **schedule_fringe_test** - script to schedule an automatic fringe test. | ||
+ | |||
+ | Inside the experiment name (/ | ||
+ | < | ||
+ | log2vex.pl | ||
+ | - tapelog evlbi | ||
+ | - EOP | ||
+ | - Sched from Skdfile | ||
+ | - Clock | ||
+ | - Compile vexfile | ||
+ | |||
+ | |||
+ | vex2db < | ||
+ | |||
+ | schedule_fringe_test [-h] -e EXPERIMENT -s STATIONS -m EMAIL [-n SCANNAME] [-t STARTTIME] [-d DURATION] | ||
+ | |||
+ | Either specify a scan or startime plus duration. | ||
+ | |||
+ | Arguments: | ||
+ | -h Show the help | ||
+ | -e EXPERIMENT | ||
+ | -s STATIONS | ||
+ | -m EMAIL Email where the results will be submitted | ||
+ | -n SCANNAME | ||
+ | -t STARTTIME | ||
+ | -d DURATION | ||
+ | </ | ||
+ | |||
+ | |||
+ | **recorrelate_fringe_test** - to correlate again a fringe test scheduled with schedule_fringe_test. | ||
+ | |||
+ | < | ||
+ | recorrelate_fringe_test -e EXPERIMENT -n SCANNAME | ||
+ | </ | ||
+ | |||
+ | |||
+ | |||
+ | |||
=====EEE(2)===== | =====EEE(2)===== | ||
Line 25: | Line 98: | ||
--version | --version | ||
-v, --verbose | -v, --verbose | ||
+ | </ | ||
+ | |||
+ | **fringeSelect.py** - script to help find you a fringe finder. Queries astrogeo rfc: | ||
+ | < | ||
+ | usage fringeSelect.py [-h] [-b BAND] [-e MIN_EL] [-f MIN_FLUX] | ||
+ | | ||
+ | |||
+ | Provide some options for sources to use as fringe finders. Based on RfC | ||
+ | catalogue. | ||
+ | |||
+ | positional arguments: | ||
+ | timeStart | ||
+ | =' | ||
+ | duration | ||
+ | stations | ||
+ | |||
+ | optional arguments: | ||
+ | -h, --help | ||
+ | -b BAND, --band BAND The band you are searching for, one of l, s, c, m, x, | ||
+ | u, k, q. If not specified will default to using C-band | ||
+ | fluxes as priority. | ||
+ | -e MIN_EL, --min-el MIN_EL | ||
+ | The minimum elevation to consider a source being ' | ||
+ | Defaults to 20. | ||
+ | -f MIN_FLUX, --min-flux MIN_FLUX | ||
+ | The mimimum flux density of sources to consider. | ||
+ | Defaults to 1.0 Jy | ||
</ | </ | ||
Line 70: | Line 170: | ||
Run it in parallel: | Run it in parallel: | ||
'' | '' | ||
+ | |||
+ | **parsePIletter.py** - reads a piletter and adds station comments therein into the feedback database. Should be run for every experiment. | ||
+ | < | ||
+ | usage: parsePIletter.py [-h] [-s SESSION] PIfile | ||
+ | |||
+ | This script will attempt to read a piletter and insert the station specific | ||
+ | comments into the feedback DB. | ||
+ | |||
+ | positional arguments: | ||
+ | PIfile | ||
+ | |||
+ | optional arguments: | ||
+ | -h, --help | ||
+ | -s SESSION, --session SESSION | ||
+ | The session name in format xxxYY (e.g. oct17) | ||
+ | </ | ||
+ | |||
+ | **ysfocus.py** - Sets the MOUNT field in the ANTENNA table correctly for Yebes/ | ||
+ | < | ||
+ | usage: ysfocus.py [-h] < | ||
+ | |||
+ | Change the MOUNT field in the ANTENNA table for Yebes to ' | ||
+ | It allows tConvert to put MNTSTA=4 into the FITS AN table to handle the | ||
+ | parallactic angle correction for Ys's Nasmyth focus correctly. Also, changes | ||
+ | Hobart X_YEW to X-YEW expected by tConvert (MNTSTA=3). | ||
+ | |||
+ | positional arguments: | ||
+ | msdata | ||
+ | |||
+ | optional arguments: | ||
+ | -h, --help | ||
+ | --version | ||
+ | </ | ||
+ | |||
+ | |||
+ | **pipelet.py** - Parses the pipe letter (exp.pipelet) to be sent to the PI after pipelining the data. | ||
+ | < | ||
+ | usage: pipelet.py [-h] [-v] [-o OUTPUT] [-c CREDENTIALS] [-u USERNAME] [-p PASSWORD] | ||
+ | | ||
+ | |||
+ | Creates a .pipelet file in the current directory. This letter is the content that must | ||
+ | be sent to the PI after pipelining an experiment. It contains information about the | ||
+ | pipeline output and the credentials to access the data. | ||
+ | It takes the credentials from the username_password.auth file that should be placed in | ||
+ | the current directory (otherwise specify its file, or the username and password as parameters). | ||
+ | The user must provide the following information: | ||
+ | - The experiment name (case insensitive). | ||
+ | - Who are you (the Support Scientist). Type your surname. | ||
+ | |||
+ | positional arguments: | ||
+ | experiment | ||
+ | jss JIVE Support Scientist doing the post-processing (your surname). | ||
+ | |||
+ | optional arguments: | ||
+ | -h, --help | ||
+ | -v, --version | ||
+ | -o OUTPUT, --output OUTPUT | ||
+ | Output directory where the file {experiment}.pipelet will be store | ||
+ | (by default the current directory) | ||
+ | -c CREDENTIALS, | ||
+ | Auth file containing the username and password in its name (required | ||
+ | if *.auth is not in current directory). | ||
+ | -u USERNAME, --username USERNAME | ||
+ | Username to access the data (required if no credential file exists) | ||
+ | -p PASSWORD, --password PASSWORD | ||
+ | Password to access the data (required if no credential file exists) | ||
+ | |||
+ | </ | ||
+ | |||
+ | =====Pipe (jop83)===== | ||
+ | |||
+ | **ampcal.sh** - Run in ' | ||
+ | |||
+ | **antab_check.py** - tool to do some basic sanity checking of antab information: | ||
+ | Do you have antabs/ | ||
+ | Also plots the tsys if requested. | ||
+ | < | ||
+ | usage: antab_check.py [-h] [-p] [-s STATION] [-i] [-l] [-f] | ||
+ | |||
+ | Some basic sanity checking for ANTAB files (all uvflgs/ | ||
+ | current directory, does INDEX line appear to have right number of | ||
+ | definitions? | ||
+ | |||
+ | optional arguments: | ||
+ | -h, --help | ||
+ | -p, --plot | ||
+ | -s STATION, --station STATION | ||
+ | Choose to only plot one station (default is to plot | ||
+ | all). Use 2 letter code (e.g. Ef). Auto adds -p,-i if | ||
+ | not given | ||
+ | -i, --no-pause | ||
+ | -l, --no-flag-line | ||
+ | -f, --no-flagged-data | ||
+ | Does not plot flagged data (data at -2) | ||
+ | </ | ||
+ | |||
+ | **antabfs_interpolate.py** - Modifies an ANTAB file to write more Tsys values when the time separation between them is too long. | ||
+ | < | ||
+ | usage: antabfs_interpolate.py [-h] [-v] [-p] [-o OUTPUTFILE] [-tini STARTIME] [-tend ENDTIME] antabfile int | ||
+ | |||
+ | |||
+ | Given an ANTAB file, it creates a new one with more Tsys values that are interpolated from the | ||
+ | given ones. This will fill gaps when the time separation between Tsys measurements is too long | ||
+ | (and e.g. AIPS starts flagging scans because there is not Tsys information for them). | ||
+ | |||
+ | IMPORTANT CONSIDERATIONS: | ||
+ | Note that it interpolates data with a smooth function, avoiding outliers or zero values. | ||
+ | However, it does not consider scan boundaries, so if Tsys are recorded in different sources | ||
+ | it can introduce biases. Therefore, it assumes that the Tsys should not change quickly | ||
+ | (e.g. no change of sources). | ||
+ | |||
+ | positional arguments: | ||
+ | antabfile | ||
+ | int The interval (in seconds) between the final Tsys measurements | ||
+ | |||
+ | optional arguments: | ||
+ | -h, --help | ||
+ | -v, --version | ||
+ | -o OUTPUT, --output OUTPUT | ||
+ | Output filename. By default same as antabfile. | ||
+ | -p, --plot | ||
+ | -tini TINI Starttime of the Tsys measurements. In case you want to modify it from the original file. It will extrapolate the earliest Tsys original values. The format must be as DOY/ | ||
+ | -tend TEND Ending time of the Tsys measurements. In case you want to modify it from the original file. It will extrapolate the latest Tsys original values. The format must be as DOY/ | ||
+ | </ | ||
+ | |||
+ | |||
+ | **antabfs_nominal.py** - makes a nominal (based on expected SEFD only) antabfs file for a telescope that did not provide antabfs: | ||
+ | < | ||
+ | usage: antabfs_nominal.py [-h] [-v] [-b BAND] [-d DURATION] [-fr FREQRANGE] | ||
+ | [-s SEFD] [-i INTERVAL] [-sb SUBBANDS] | ||
+ | antenna experiment start | ||
+ | |||
+ | Writes a nominal SEFD ANTAB file. Gain will be set to 1/SEFD, and all Tsys to | ||
+ | 1.0. It will overwrite any previous antab file in the current path. | ||
+ | antabfs_nominal.py uses the SEFD information from sefd_values.txt to compute | ||
+ | the nominal values. Creates (or overwrites) a file called | ||
+ | < | ||
+ | from the user. | ||
+ | |||
+ | positional arguments: | ||
+ | antenna | ||
+ | Ro7 Ro3) | ||
+ | experiment | ||
+ | start Start time (DOY/HH:MM, YYYY/ | ||
+ | YYYY/ | ||
+ | |||
+ | optional arguments: | ||
+ | -h, --help | ||
+ | -v, --version | ||
+ | -b BAND, --band BAND Observed band (in cm). REQUIRED unless SEFD provided | ||
+ | -d DURATION, --duration DURATION | ||
+ | Duration of the experiment (in hours). Default: 24 h | ||
+ | -fr FREQRANGE, --freqrange FREQRANGE | ||
+ | Frequency range where the ANTAB is applicable (lower | ||
+ | and upper limit, in MHz). Default 100,100000 (please, | ||
+ | do not use spaces between the numbers). | ||
+ | -s SEFD, --sefd SEFD SEFD to be used (optional). Default values are loaded. | ||
+ | -i INTERVAL, --interval INTERVAL | ||
+ | Interval between Tsys measurements (in min). Default: | ||
+ | 0.5 | ||
+ | -sb SUBBANDS, --subbands SUBBANDS | ||
+ | Number of subbands in the experiment. Default: 8 = | ||
+ | L1|R1 L2|R2 ... L8|R8 | ||
+ | </ | ||
+ | |||
+ | **sftpvlbeer** - for the very lazy (y/w - Jay) | ||
+ | |||
+ | '' | ||
+ | |||
+ | **uvflgall.csh** - makes a uvflgfs file from any .log files in the current directory. Also changes flagr line to make uvflg.pl happier. | ||
+ | |||
+ | **comment_tasav_file.py** - tool to create the comment file to be located in the pipeline output before running the feedback.pl script (and after having run the pipeline): | ||
+ | |||
+ | < | ||
+ | usage: comment_tasav_file.py [-h] [-v] [-oc OUTPUT_COMMENT] [-ot OUTPUT_TASAV] | ||
+ | | ||
+ | |||
+ | Creates a .comment file (in $OUT/exp directory) and a .tasav.txt file (in | ||
+ | $IN/exp). Given the default templates, it customizes them to include the basic | ||
+ | data from the given experiment. The script takes the information from | ||
+ | different locations (ccsbeta, pipe $IN and $OUT directories). The EVN Pipeline | ||
+ | must have been run before calling this script. | ||
+ | |||
+ | positional arguments: | ||
+ | experiment | ||
+ | write {exp}_number (e.g. ev100_1 | ||
+ | |||
+ | optional arguments: | ||
+ | -h, --help | ||
+ | -v, --version | ||
+ | -oc OUTPUT_COMMENT, | ||
+ | Output directory where the file {experiment}.comment | ||
+ | will be saved (by default in $OUT/ | ||
+ | -ot OUTPUT_TASAV, | ||
+ | Output directory where the file {experiment}.tasav.txt | ||
+ | will be saved (by default in $IN/ | ||
+ | |||
+ | </ | ||
+ | All additional data are taken from different files: the script access ccsbeta to get the epoch of the observation (from the MASTER_PROJECT.LIS file), and it also reads the pipeline input file and the pipeline output .SCAN and .DTSUM files. The default output will be in $OUT/ | ||
+ | |||
+ | |||
supportsci/tools.1532606138.txt.gz · Last modified: 2018/07/26 11:55 by jay