carbon+star+reduction

This page will document the data reduction history for a carbon star project.

First off, we need to know which stars were observed by PTI. Some time ago, I compiled a master list of targets observed by PTI (or, at least, that **should** have been observed) based on accepted observing proposals. This master list is complete as of 2004-11-15, which is insufficient for other types of target (eg. giant stars) but carbon star observing should have been complete by then. (Unfortunately, this master list is technically not public, either, so I will not post it here.)
 * 1. Inventory of Target Stars.**

Additionally, a list of S Stars was also observed and shall be included in this reduction as "somewhat similar" objects.

Total number of targets: 58 carbons, 21 S-type, plus Miras: 1 carbon and & S-type.

I used the master list to generate the wbCalib calibration script, appending the "standard" list of calibrators to the end. This list I found in my calscript for supergiants (located on my drive at D:\projects\pti\data\07_supergiants\wbCalib_data). (Important reminder here: save calscript text files in Unix, not Windows, format.)

This is pretty straightforward, again copying liberally from the supergiants project. From the Ubuntu shell: code ~/mscSoftware/bin/wbCalib -nojitterCorrection /mnt/hgfs/patronella/projects/pti/data/09_carbon_S-type/wbCalib/CS-list_090615a.cs /mnt/hgfs/patronella/projects_aux/ptidata/98/*.sum /mnt/hgfs/patronella/projects_aux/ptidata/99/*.sum /mnt/hgfs/patronella/projects_aux/ptidata/100/*.sum /mnt/hgfs/patronella/projects_aux/ptidata/101/*.sum /mnt/hgfs/patronella/projects_aux/ptidata/102/*.sum /mnt/hgfs/patronella/projects_aux/ptidata/103/*.sum /mnt/hgfs/patronella/projects_aux/ptidata/104/*.sum /mnt/hgfs/patronella/projects_aux/ptidata/105/*.sum /mnt/hgfs/patronella/projects_aux/ptidata/106/*.sum /mnt/hgfs/patronella/projects_aux/ptidata/107/*.sum /mnt/hgfs/patronella/projects_aux/ptidata/108/*.sum > /mnt/hgfs/patronella/projects/pti/data/09_carbon_S-type/wbCalib/wbOut_090615a.txt
 * 2. Execute the wbCalib command.**

code  (yes this is a single command line) and the result should pop out in the wbOut file.  code ~/mscSoftware/bin/wbCalib /mnt/hgfs/patronella/projects/pti/data/09_carbon_S-type/wbCalib/CS-list_090615a.cs /mnt/hgfs/patronella/projects_aux/ptidata/98/*.sum /mnt/hgfs/patronella/projects_aux/ptidata/99/*.sum /mnt/hgfs/patronella/projects_aux/ptidata/100/*.sum /mnt/hgfs/patronella/projects_aux/ptidata/101/*.sum /mnt/hgfs/patronella/projects_aux/ptidata/102/*.sum /mnt/hgfs/patronella/projects_aux/ptidata/103/*.sum > /mnt/hgfs/patronella/projects/pti/data/09_carbon_S-type/wbCalib/wbOut_090615a-A.txt
 * Interesting problem: ran into the stack smashing error as noted in the wiki. Had to recompile as recommended but needed to do so under 4.2 versions of gcc and g++.
 * Second problem: the full command line noted above entails a data collection that overwhelms wbCalib . It needs to be broken into two pieces

~/mscSoftware/bin/wbCalib /mnt/hgfs/patronella/projects/pti/data/09_carbon_S-type/wbCalib/CS-list_090615a.cs /mnt/hgfs/patronella/projects_aux/ptidata/104/*.sum /mnt/hgfs/patronella/projects_aux/ptidata/105/*.sum /mnt/hgfs/patronella/projects_aux/ptidata/106/*.sum /mnt/hgfs/patronella/projects_aux/ptidata/107/*.sum /mnt/hgfs/patronella/projects_aux/ptidata/108/*.sum > /mnt/hgfs/patronella/projects/pti/data/09_carbon_S-type/wbCalib/wbOut_090615a-B.txt code after the wbCalib command, one can optionally add the -nojitterCorrection command, but this does not improve data quantity, though it does sacrifice data quality. Here are the output files from wbCalib: separate  and combined

The "-B" wbOut file seems to be a bit larger than expected; upon inspection, there are an inordinately large number of "# Warning: jitter 0 out of range..." lines, which I eliminated with the usual grep magic command: grep -v 'jitter 0' wbOut_090615a-B0.txt > wbOut_090615a-B.txt.

3a. Incorrect baseline used?** One possible pitfall is that wbCalib grabs the wrong baseline. This will manifest itself in the data as a cascade of "measured/expected delay mismatch" warnings. However, these come in two flavors: code
 * 3. Inspect the wbCalib data.
 * 1) Warning: at 52228.5147, -0.192688 m measured/expected delay mismatch on calibrator HDC50692
 * 2) Warning: at 52228.5421, -0.19252 m measured/expected delay mismatch on calibrator HDC50692
 * 3) Warning: at 52228.5713, -0.192286 m measured/expected delay mismatch on calibrator HDC50692
 * 4) Warning: at 52228.5234, -0.193021 m measured/expected delay mismatch on calibrator HDC56537
 * 5) Warning: at 52228.5512, -0.192811 m measured/expected delay mismatch on calibrator HDC56537
 * 6) Warning: at 52228.5556, -0.192737 m measured/expected delay mismatch on target HDC55284

code This is just a true glitch in the baseline, nothing to worry about. But something to look at more closely: code
 * 1) Reject: at 54109.3582 sysVis estimates between calibrators HDC56537 and HDC58551 are inconsistent at the 5.8 sigma level
 * 2) No suitable calibration found for 54109.3582
 * 3) Warning: at 54109.3440, -4.58009 m measured/expected delay mismatch on calibrator HDC56537
 * 4) Warning: at 54109.3621, 0.927776 m measured/expected delay mismatch on calibrator HDC56537
 * 5) Warning: at 54109.3520, 2.15689 m measured/expected delay mismatch on calibrator HDC58551
 * 6) Warning: at 54109.3539, 2.68763 m measured/expected delay mismatch on calibrator HDC58551
 * 7) Warning: at 54109.3711, 7.8672 m measured/expected delay mismatch on calibrator HDC58551
 * 8) Warning: at 54109.3729, 8.40993 m measured/expected delay mismatch on calibrator HDC58551
 * 9) Warning: at 54109.3985, 11.847 m measured/expected delay mismatch on calibrator HDC79452

code This this case, the numbers march steadily, and a second telling point is the inconsistency of the calibrator sysVis estimates. Let's go look and see: (1) an inspection of the various hits for "54109" in the wbCalib output file shows to things - (a) consistent offsets of roughly -0.22m before 54109.34, and (b) increasing quantites thereafter. Possibly a baseline switch in the middle of the night? (2) We need to convert 54109 from JD to PTI-night-number. Using the Excel spreadsheet, we can take a stab at this. Wait, this doesn't really work, looks like it's way outside the range of the set nights; a quick edit to that program file indicates night #10 in 2007. Using the [|converter at the USNO] (and adding 2,400,000), we can find that 08 Jan 2007. (3) We can now inspect the log file for that night: there is no 107008.log; a look at 107007.log shows nothing obvious, but 107009.log has the following remarks: code 5:56:46: This is Night 107009, KMR and BT here. Finished changing out 1-in mirrors in all 3 huts. Will test on sky if seeing permits. 5:56:58: Path= /home/night3/07009 5:57:20: In K-10ms, NS baseline. 5:57:48: Adjusted focus on all 3 huts, and co-aligned all hut optics. 6:18:08: New Starlist sent to Sequencer.

...

7:46:46: Considering the seeing, and the dust in the air, I'd say the data looks good. ST photon counts are well balanced. Will switch to NW next to do a direct compare on this same cluster from the last time we took data on it. 7:47:42: Star:   HDC55284 : Start:  7:41:33, Fringe:  7:42:53, Locks: 145/0130, HA:  0.02, Zenith: 18.0, V2: 0.124, Delay:  4851.60, Photons:   2518.7, BG: -11.07, SpecV2:  0.880, SpecPhotons:      8.6, Jitter: 1.260 Seeing: 1.63 /  1.36 StPhot:  12358.7 /  11547.0 7:57:02: Booted up in NW baseline, K10ms 8:07:26: New Starlist sent to Sequencer.

...

code A-ha! This is the source of the problem - a little detective work pays off here. How to fix? We'll need to split the *.sum and *.spec files into two separate halves (and also bisect the *.baseline file for 107009) and re-run the wbCalib command in step 2 above.

Fortunately this appears to be the only instance of this happening.

Sometime wbCalib reports calibrators in conflict. Good example: HD56537 and HD58551 look to be in conflict for the nights of 107007 and 107009  (inspect the "Incoherent Spec V2" on page 5): code
 * 3b. Dueling calibrators (of which some are bad?)**
 * 1) Reject: at 54109.3582 sysVis estimates between calibrators HDC56537 and HDC58551 are inconsistent at the 6.5 sigma level
 * 2) No suitable calibration found for 54109.3582

code Which one is right? Hard to say. My gut feeling is to go with the smaller star (HD58551 @ 0.34mas) but an examination of the PS file of the night's results show that data to be very ratty. Plus, a look at the PS of the sedFit for HD56537 (@ 0.798mas) reveals it to be a very good fit. Hence, an update of the wbCalib *.cs file comments out HD58551.

Next most obvious culprit: searching through the wbCalib output files and looking for the star that turns up the most frequently with reject lines (searching on "are inconsistent at the" works well), we find HDC6920 turns up 32 times with a variety of other calibrators - not a good sign. Take a peek at the nightly report PS files to see if the star is behaving badly - find them using: code $ grep -c 'HDC6920' 10[0-9]/ps/*.sum.ps | grep -v ':0' code (This does not trawl through the 98 and 99 data.) JD 52892 = 09 Sep 2003 = 103253 looks like a good culprit from the wbCalib output file, but it's not obvious upon inspection. However, HDC7964 and HDC7034 behave strangely relative to HD6920 on this night...(?) This is particularly of interest given the use of HDC7964/7034 as calibrators on the [|recent PTI paper for HD9939's orbit]. A review of overlapping nights does not clarify things: 101237 looks odd. 102304 looks fine, as does 105295, 105296, 105303.

An examination of the relative rankings of HDC6920/7964 in the [|PTI calibrator paper] shows no strong preference for the other. However, in the interest if discretion before valor, we will retire HDC6920 for now.

The next most common 'hit' is HDC48805, but upon inspection of the nightly PS files, that appears to be a function of this object frequently being used at end-of-night.

We can now grep the wbOut files for "No suitable calibration found..." and see if there's a particularly painful spot in terms of no calibrator object being found on a certain night, due to a lack of calibrator information in the calscript (eg. a calibrator used on a given night that isn't on the 'standard' list). For example, night 52620=102346 lists this error at total of 13 times (!). Inspection of this PS file reveals it to merely be a particularly ratty night. The same appears to be true of 52262=101353. 52144=101235 looks like there is a HA-dependent effect of the V2 response that I'm going to just ignore and let cancel out the data, rather than take a chance on the data.
 * 3c. Unavailable calibrators?**

So, more or less, there's no obvious place where missing calibration stars is preventing us from getting (good) target data.

//The curious case of the unavailable 2008 data// It looks like some data was missing from the wbCalib reduction, associated with 2008 for HIP92194(=DR Ser; nights 108176, 108177, 108184, 108185). These are nights associated with the IONIC combiner and their "*.slowscan.sum" files do not appear to dribble through the wbCalib pipeline.

So, how many stars & associated data points did we get out the back end of the wbCalib pipeline? Quite a few, according to the tally in this Excel file.
 * 4. The score**.

Now that the wbCalib reduction is done, it should be (in a perfect world) easy to do the nbCalib reduction as well - just modify the command lines above, changing 'wbCalib' to 'nbCalib' and '.sum' to '.spec': code ~/mscSoftware/bin/nbCalib /mnt/hgfs/patronella/projects/pti/data/09_carbon_S-type/nbCalib/CS-list_090615a.cs /mnt/hgfs/patronella/projects_aux/ptidata/98/*.spec /mnt/hgfs/patronella/projects_aux/ptidata/99/*.spec /mnt/hgfs/patronella/projects_aux/ptidata/100/*.spec /mnt/hgfs/patronella/projects_aux/ptidata/101/*.spec /mnt/hgfs/patronella/projects_aux/ptidata/102/*.spec /mnt/hgfs/patronella/projects_aux/ptidata/103/*.spec > /mnt/hgfs/patronella/projects/pti/data/09_carbon_S-type/wbCalib/nbOut_090617a-A.txt
 * 5. nbCalib reduction**.

~/mscSoftware/bin/nbCalib /mnt/hgfs/patronella/projects/pti/data/09_carbon_S-type/nbCalib/CS-list_090615a.cs /mnt/hgfs/patronella/projects_aux/ptidata/104/*.spec /mnt/hgfs/patronella/projects_aux/ptidata/105/*.spec /mnt/hgfs/patronella/projects_aux/ptidata/106/*.spec /mnt/hgfs/patronella/projects_aux/ptidata/107/*.spec /mnt/hgfs/patronella/projects_aux/ptidata/108/*.spec > /mnt/hgfs/patronella/projects/pti/data/09_carbon_S-type/wbCalib/nbOut_090617a-B.txt code No change should be necessary for the calscript, CS-list_090615a.cs. Let's see if it works.

It didn't, but only because I forgot to save the text file containing the commands in Unix, not Windows format. Argh! Simple fix, though.

Looks like with that fix, the nbCalib reduction proceeds just fine. Extra unexpected bonus: most of the data is 5-channel K-band, though inspection of the output file reveals that there is some 4-channel H-band, and even some 'high-resolution' 9- and 10-channel K-band on 7 targets.

Using a simple grep command: code grep HDC209890 wbOut_090615a-all.txt > HDC209890.wbOut code the various relevant bits from the wbCalib (and nbCalib) data files can be extracted into bite-sized smaller files. The data count Excel file above can be handily adapted to generate a quick list of these commands which results in command line files for wbCalib  and nbCalib.
 * 6. Slice & dice the data.**

Using the handy tool wbFitSingle (also provided by code master extraordinaire Andy Boden). We can once again take the PTI identifiers in the first column of the above data count Excel file and use them to generate a repeating series of commands along the lines of: code ./wbFitSingle HDC209890.wbOut >> fit_full.txt cp fit.out HDC209890.wbOut.fit ./wbFitSingle HDC209890.wbOut 1 1.4 1.99 >> fit_H.txt ./wbFitSingle HDC209890.wbOut 1 1.98 2.4 >> fit_K.txt
 * 7. Do some simple uniform disk size fitting.**

code which is useful. As a result, the following files are generated. This can be converted easily into a easier to read (and sortable & otherwise manipulable) Excel file.

Similarly, nbFitSingle does the same job but speaks nbOut rather than wbOut (including that funky variable line-length format associated with the varying numbers of spectral channels at any given time - usually 5 for K-band, sometimes 4 for H-band, sometimes twice those numbers (or more!) for 'high resolution' mode): code ./nbFitSingle HDC209890.nbOut >> fit_full.txt cp fit.out HDC209890.nbOut.fit ./nbFitSingle HDC209890.nbOut 1 1.525 1.575 >> fit_H1.txt ./nbFitSingle HDC209890.nbOut 1 1.575 1.625 >> fit_H2.txt ./nbFitSingle HDC209890.nbOut 1 1.625 1.675 >> fit_H3.txt ./nbFitSingle HDC209890.nbOut 1 1.675 1.725 >> fit_H4.txt ./nbFitSingle HDC209890.nbOut 1 1.95 2.05 >> fit_K1.txt ./nbFitSingle HDC209890.nbOut 1 2.05 2.15 >> fit_K2.txt ./nbFitSingle HDC209890.nbOut 1 2.15 2.25 >> fit_K3.txt ./nbFitSingle HDC209890.nbOut 1 2.25 2.35 >> fit_K4.txt ./nbFitSingle HDC209890.nbOut 1 2.35 2.45 >> fit_K5.txt ./nbFitSingle HDC209890.nbOut 1 1.95 2.45 >> fit_K0.txt ./nbFitSingle HDC209890.nbOut 1 1.525 1.725 >> fit_H0.txt

code A bit more work but more results (er, at lower SNR, though).

This is a little more tricky and can stand to use more appropriate center-to-limb variation profiles than a simple uniform disk (eg. as an example, see [|the recent Paladini paper], noting that these are dynamic models; static models are what are necessary for this data set).
 * 8. Do more appropriate size fitting.**

A dicey proposition - these objects tend to be variable. But a fair stab at this is reasonable. To automate this process, we have a script called getAllPhotometry.batx, which in turn invokes <span style="font-family: 'Courier New',Courier,monospace;">getC<span style="font-family: 'Comic Sans MS',cursive;"> al and a python script called GCPD2.py. code fbolFormat DR_SER -strom -geneva -2Mass -longWL -CIO > foo3
 * 9. Collect photometry on these objects.**

code This script automatically queries SIMBAD for photometry information, and the [|GCPD].

X.a. Spectral types** Reference (a) General Catalog of S Stars, second edition (Stephenson 1984) and (b) General Catalog of galactic Carbon stars, 3d Ed. (Alksnis+ 2001); in particular, (b) contains spectral types for carbon stars from Yamashita (1972,1975) even if original PDFs cannot be found on ADS.
 * X. Comments on supporting information.