User notes for FERRET Rob Scott Started March 2002 with version 5.40 Latest additions: June 2004 (now using v. 5.60 on Redhat linux). Table of Contents: Part I How to?! 1. Regridding: Climatologies and Anomalies, @ASN, etc. 2. Defining/creating Time axes 3. Converting abstract variables to gridded variables 4. Running LINUX commands 5. Read many files into ferret (getting file list from LINUX ls *.nc) 7. Nest repeats 8. Line continuation 9. IF THEN ELSE 10. name lengths 11. RESHAPE 12. CONTOUR labels 13. definite integrals @DIN 14. ZAXREPLACE 15. MODULO axis not recognized 16. XSEQUENCE, and how do I get data OFF a grid!! Part II Problems with ferret, documentation, or demos! 1. Problem with isize of string array: 2. Problem with interpolating filler @FLN 3. Problem with DEFINE AXIS/FROM_DATA 4. Omission in Ch3 Sec2.5. IF-THEN logic ("masking") 5. (only old) constant_array_demo crashes (new one doesn't) 6. ferret540 causes edit_data_file_demo.jnl Part III Warnings and pitfalls!! 1. Caution using indefinite integral, @IIN 2. Warning about REPEAT 3. **ERROR: illegal limits: (C022,V001) is not in the range I=2 Axis extremes are I=1, see XSEQUENCE above 4. **ERROR: ga_ts_total_11_v5.dat1_v5.dq: Success LIST /FILE=ga_ts_total_11_v5.dat/FORMAT=(2(1X,E12.5))/CLOBBER influx_total/area, influx_11/area1 Command file, command group, or REPEAT execution aborted 5. Buffer overflow on output Program terminated by fatal I/O error 6. STOP: is_trans:ndim = 0 Warning: Floating underflow occurred 7. Segmentation fault related to character variables 8. Strange error resulting from regridding ************************************************************************* **************************** Part I ************************************ ************************************************************************* 1. Climatologies and Anomalies Note: References: For creating climatologies, see modulo regridding, Ferret Users' Guide, Chapter 4.5 For creating anomalies, see FAQ: http://ferret.wrc.noaa.gov/Ferret/FAQ/analysis/climatologies.html Experience: -------------------------------------------------------------------------------- 1.1. Climatologies 1.1.1 ferret540 Documentation Bug?? To avoid the misalignment problem, must first regrid raw data to center of month values (since modulo regridding uses unweighted averaging). It doesn't work to follow their example in the text: yes? LET test_var_centered = test_var[GT=month_reg] or for my data: yes? let myvar_midmon = myvar[GT=month_reg] gives an empty variable of only 12 months long (even though tke has 96 time values). Workaround, just create my own t-axis centered on the middle of the months: yes? define axis/T=01-OCT-1993:31-DEC-2000:`365.2425/12`/UNITS=days/edges taxis_midmon yes? let myvar_midmon = myvar[GT=taxis_midmon] 1.1.2 But then, to get the anomalies, it is important NOT to follow do: LET myanom = myvar_midmon - myclim[GT=month_reg@ASN] This will associate the JAN climatology with the first month of the time axis taxis_midmon. Just use your center of month taxis, sans @ASN, and it seems to work fine: LET myanom = myvar_midmon - myclim[GT=taxis_midmon] -------------------------------------------------------------------------------- 1.2. ASN regridding: 1.2.1 Warning: Specify the source variable range using indices (using x,y,z,t range may lead to problems if that range is outside the range of the axis of the target). 1.2.2 Warning: Seems to be a problem referring to the dynamic grids via the variable name. However it works with the axis name: this works: let MHFLX = SST_MF[X=141.5E:179.5E] * V_G[I=1:39,GX=X_AXIS_MF@ASN] but this doesn't: let MHFLX = SST_MF[X=141.5E:179.5E] * V_G[I=1:39,GX=SST_MF@ASN] 1.2.3 Warning: Very significant errors result in a subtle way if the range of x is not specified for all variables in an expression, e.g. the following gives different answers than above: let MHFLX = SST_MF * V_G[I=1:39,GX=SST_MF@ASN] ******************************************************************************** 2. Defining/creating Time axes 2.1 What if there is a contradiction within beginning:end:step? Looks like ferret540 respects the beginning and step, and adjusts the end. e.g. DEFINE AXIS/T="15-NOV-1992:14:05:33":"17-MAY-1999:23:10:57":`365.2425/12`/UNITS=days mytaxis gave a penultimate point: L = 79, T = 17-MAY-1999 16:09:23 Also added an extra point in June, L=80. 2.2 How to convert from a data variable to a grid axis? Simply use /FROM_DATA e.g. DEFINE AXIS/T/FROM_DATA/NAME=taxis_qiu/UNITS=years date But note that it's not a calendar date, by a time. I don't know how to convert to calendar date. ******************************************************************************** 3. Converting abstract variables to gridded variables Use RESHAPE function. see Ch3 Sec2.3.23. RESHAPE ******************************************************************************** 4. Running LINUX commands Use SPAWN. ******************************************************************************** 5. Read many files into ferret (getting file list from LINUX ls *.nc) ! create a string array, "filelist", that contains the output of LINUX ! command "ls *.nc" LET filelist = {spawn:"ls *.nc"} ! repeatedly access this array as follows ! (bit awkward, have to define the temporary string, "a") REPEAT/i=1:`filelist,return=isize` (LET a = filelist[i=`i`]; SAY `a`) ! practical usage !REPEAT/i=1:`filelist,return=isize` (LET a = filelist[i=`i`]; SET DATA `a`; GO my_go_file) ******************************************************************************** ******************************************************************************** 7. Nest repeats This doesn't work: repeat/i=1:2/j=2:3 Have to "nest the repeats", like this: repeat/L=2:3 repeat/k=1:3 (list k+2-1; list k+1-2) Subtleties: This doesn't work: repeat/L=2:3 (repeat/k=1:3 (list k+2-1; list k+1-2)) yet this does: repeat/L=2:3 (repeat/k=1:3 (list k+2-1; list k+1-2);) ************************************************************************* 8. Line continuation use backslash \ ************************************************************************* 9. IF THEN ELSE Warning: END IF doesn't work, must use ENDIF Warning: very fussy about the syntac: grave accents needed: `$1 LT -90` ; but not: $1 LT -90 space needed: `$1 LT -90` THEN ; but not: `$1 LT -90`THEN ************************************************************************* 10. name lengths From trial and error, seems grid names are restricted to 16 characters or less, and resummably applies to all names. ************************************************************************* 11. RESHAPE(A,B) Be careful with reshape! It's more powerful than regridding tools. It's not clear what order the values are taken from A(i,j,k,l) when it's "wrapping" it onto the grid of B. Does it follow the listing order? 12. CONTOUR labels Contour labels are automatically added to contour plots by default. If you don't see them, perhaps there isn't enough room to plot them. Try supplying more room via a combination of the following: a) reduce the number of significant digits of the label with the final levels parameter. CONTOUR/levels=(10,20,1,-1) the "-1" indicates no decimal points. b) smooth the field c) reduce number of contour levels, continuing above e.g. contour/levels=(10,20,2,-1) d) reduce size of label (I can't get this to work, but in theory...) PPL CONSET 0.04 then CONTOUR/levels=(10,20,1,-1) Note that in general, to control number of decimal points of contour labels, supply integer "n" below: (n = -1, no decimal points, n= -3, no labels at all): CONTOUR/levels=(low,hi,delta,n) ************************************************************************* 13. definite integrals @DIN If you define a variable as: SET GRID mygrid LET myvar = 1 LIST myvar[x=@DIN] will be wrong! it will just be 1, regardless of the axis length. see my_din_demo.jnl for an example, and other properties about @DIN. 14. ZAXREPLACE I find identical results with: ZAXREPLACE(V,ZVALS,z[GZ=myzax]) ZAXREPLACE(V,ZVALS,z[GZ=myzax@AVE]) If find differences near missing values with: ZAXREPLACE(V,ZVALS,z[GZ=myzax]) ZAXREPLACE_AVG(V,ZVALS,z[GZ=myzax]) As a general recommendation, fill missing values in V first!! 15. MODULO axis not recognized use SET AXIS/MODULO axis_name 16. XSEQUENCE Often you have data on a grid, and you want ferret to forget about that grid. Otherwise, later you get weird error messages like: **ERROR: illegal limits: (C022,V001) is not in the range I=2 Axis extremes are I=1 Use XSEQUENCE, or one of the other three similar functions to put it on an abstract grid. ************************************************************************* **************************** Part II *********************************** ************************************************************************* 1. Problem with isize of string array: let names = {spawn:"ls ssh10d199210*"} yes? list names VARIABLE : {SPAWN:"ls ssh10d199210*"} SUBSET : 6 points (X) 1 / 1:"ssh10d19921004.nc" 2 / 2:"ssh10d19921009.nc" 3 / 3:"ssh10d19921014.nc" 4 / 4:"ssh10d19921019.nc" 5 / 5:"ssh10d19921024.nc" 6 / 6:"ssh10d19921029.nc" yes? say `names,return=isize` !-> MESSAGE/CONTINUE 2000 2000 ************************************************************************* 2. Problem with interpolating filler @FLN "Ch3 Sec2.4.24. @FLN:n-linear interpolation filler The transformation @FLN:n fills holes in variables with a linear interpolation from the nearest non-missing surrounding point. n specifies the number of points beyond the edge of the indicated axis limits to include in the search for interpolants (default n = 1). Unlike @FAV, @FLN is sensitive to unevenly spaced points and computes its linear interpolation based on the world coordinate locations of grid points. Any gap of missing values that has a valid data point on each end will be filled, regardless of the length of the gap. However, when a sub-region from the full span of the data is requested sometimes a fillable gap crosses the border of the requested region. In this case the valid data point from which interpolation should be computed is not available. The parameter n tells Ferret how far beyond the border of the requested region to look for a valid data point. See General Information (p 82) for important details about this transformation." This just isn't how it works! n is the number of points to look along the indicated axis for a valid point, regardless of the "border of the requested region". ************************************************************************* 3. Problem with DEFINE AXIS/FROM_DATA The example from the manual doesn't work: yes? DEFINE AXIS/FROM_DATA/NAME=axis_name expr You must supply a direction. ************************************************************************* 4. Omission in Ch3 Sec2.5. IF-THEN logic ("masking") EQ is allowed for == (i.e. FORTRAN .EQ.) ************************************************************************* 5. constant_array_demo crashes The following lines crapped out: USE coads_climatology LET my_sst_winter = SAMPLEL({1,2,12},sst[Y=0:90N]) LET my_sst_summer = SAMPLEL({6,7,8}, sst[Y=0:90N]) But I got it to run by doing the following modification: LET my_sst_winter = SAMPLEL(sst[Y=0:90N],{1,2,12}) LET my_sst_summer = SAMPLEL(sst[Y=0:90N],{6,7,8}) ************************************************************************* 6. ferret540 causes edit_data_file_demo.jnl to crash yet ferret533 doesn't! ************************************************************************* **************************** Part III ************************************ ************************************************************************* 1. Caution using indefinite integral, @IIN The integral starts from the start of the base of the axis, e.g. for the time axis, it's the L=1 TBOXLO value. If this is undefined, the results is undefined everywhere! To get it started, must specify a starting time that is within a grid cell that is defined, e.g.: LET ga_kf_REY_ANOM[T="31-DEC-1992:07:27:51":15-DEC-1993@IIN] This works because: yes? list ga_kf_REY_ANOM[T=01-JAN-1992:15-DEC-1993] GA_KF_REY_MIDMON - GA_KF_REY_CLIM[GT=TAXIS_MIDMON] LATITUDE: 31N to 37N 141E 1 16-OCT-1992 / 1: .... 15-NOV-1992 / 2: .... 16-DEC-1992 / 3: .... 15-JAN-1993 / 4: -4.208E-09 and the TBOXLO is: yes? show grid/T ga_kf_REY_ANOM[T=01-JAN-1992:15-DEC-1993] GRID (G006) name axis # pts start end X_AXIS_C LONGITUDE 40 r 141E 180E Y_AXIS_ZF LATITUDE 6 r 31.5N 36.5N ABSTRACT Z 9999999 r 1 9999999 TAXIS_MIDMON TIME 111 r 16-OCT-1992 05:14 16-DEC-2001 06:55 L T TBOX TBOXLO TSTEP (DAYS) 1> 16-OCT-1992 05:14:38 30.437 01-OCT-1992 00:00:00 33512.22 2> 15-NOV-1992 15:43:55 30.437 31-OCT-1992 10:29:16 33542.66 3> 16-DEC-1992 02:13:12 30.437 30-NOV-1992 20:58:33 33573.1 4> 15-JAN-1993 12:42:28 30.437 31-DEC-1992 07:27:50 33603.53 IF you give indices L=lmin:lmax to specify the range, it goes from TBOXLO of the first indice lmin to TBOXLO of lmax+1. Alternatively you can can specify fractional grid box values by giving T=tmin:tmax. 2. Warning about REPEAT If you have too many lines in a REPEAT command, get an error that crashes ferret! Too fix this problem, simply put the commands in a GO file, and call that from within the REPEAT loop. 3. see XSEQUENCE in Part I. 4. ya can't clobber an ASCII data file! 5. might be too much stuff in a repeat (if that repeat has an output) 6. Got this one when I tried to take a time average of something that didn't involve time. 7. Segmentation fault related to character variables: Here was the problem: yes? LET names = {SPAWN:"ls /rbs0/Altimeter_data/Aviso/NRT_SLA/All_Satellites/msla_oer_merged_h_19[456]??.nc"} yes? stat names Segmentation fault Solution: I just shortened the directory path name! Apparently there is a limit to the size of character variables, and you get a segmentation fault if you exceed it with the SPAWN command. 8. Strange error resulting from regridding: Sometimes you'll get an error from defining a regrid. The solution just might be to give the grid name, as opposed a variable name on that grid. e.g. LET ssh = ssha + ssh_bar[G=ssha] might lead to weird errors, like ssh_bar[G=] undefined Try, LET ssh = ssha + ssh_bar[G=grid_anom] where grid_anom is the grid name.