wiki:FrequentlyAskedQuestions

Frequently Asked Questions

Here you can find answers to some of the most frequently asked questions about the UM and FCM. If you have a question not answered on this page, you can ask it on the UM Helpdesk


  1. General FCM Questions
    1. FCM / Subversion file status codes
    2. Where can I find more information on FCM?
  2. Code Development Issues
    1. How do I apply a global compile override?
    2. How do I apply a file specific compile override?
    3. How do I upgrade a branch from one UM version to the next?
    4. How do I create a branch at HG6.6.3 (HadGEM2-ES)?
    5. How do I resolve a branch/file conflict?
    6. How do I restore a deleted branch?
  3. Sharing code changes (fcm mkpatch)
    1. How do I apply an FCM patch received from the Met Office?
    2. How do I share my changes with the Met Office?
    3. How do I create an FCM patch?
    4. fcm-import-patch Troubleshooting
      1. can't find file to patch at input line 5
      2. cp: missing destination file operand after ....…
  4. FCM Extract ERROR and WARNING messages
    1. How do I know if the Extract process failed?
    2. vn7.1_cfg: revision keyword not found for .…
    3. ERROR: UM/control/top_level/um_shell.F90: merge - cannot resolve conflicts:
    4. Error: Timed out, host not responding
  5. Common UM compile output ERROR and WARNING messages
    1. General Guide to UM compile output
    2. gmake: * No rule to make target <routine1>.done, needed by <routine2>.done. Stop.
    3. WARNING: ac.o: duplicated targets for building
    4. Makefile:16945: warning: overriding commands for target `gather_field_ml.o'
  6. Trac Questions
    1. How do I receive notification of changes to a branch?


General FCM Questions

FCM / Subversion file status codes

When you run the command:

fcm status

or

fcm merge <branch_name>

you will see a list of the files that have been changed since the last commit. Each file will have a status code in the first column. For example:

ros@puma$ fcm status
=> svn status
M      src/control/top_level/glue_rad-rad_ctl2.F90
M      src/control/top_level/ni_imp_ctl.F90
A  +   src/control/mpp/swapable_field_mod.F90
D      src/include/other/swrdiag.h
D      src/include/other/swapable_field.h
M      src/include/argument/arg_atm_fields.h
M      src/atmosphere/boundary_layer/sf_expl.F90

The meaning of the most common status codes are as follows:

AFile scheduled for addition
CFile has conflicts. These need to be manually resolved before the working copy can be used or fcm commit run
DFile scheduled for deletion
GMerGed. FCM has resolved a conflict automatically. (I.e. Same file changed but in different places)
MFile has been modified
UFile updated with changes from the repository.
?File is not under version control
||File is missing (e.g., you moved or deleted it without using fcm).

Where can I find more information on FCM?

See the FCM User Guide on the NCAS-CMS website under UM Documentation -> Met Office Documents.


Code Development Issues

How do I apply a global compile override?

Default machine declarations are held in a config file in FCM so have a look at this first: http://puma.nerc.ac.uk/trac/UM/browser/UM/branches/dev/um/VN7.1_machine_cfg/src/configs/machines/hector-pathscale/machine.cfg

Global compiler flags are controlled by the value of %fflags64_mpp and linker flags by %ldflags64_mpp

For example to add the -g option:

Create a new text file (e.g. $HOME/umui_jobs/overrides/mach_ovrds) on PUMA and add in the following lines:

%fflags64_mpp %fflags64_mpp -pg
%ldflags64_mpp %ldflags64_mpp -pg

Here we've redeclared the value of each variable to its current value and added on the -g option.

One you have done this and saved the file, add the path and name of this override file in the 'User machine overrides' table on panel SubModel Independent -> Comp and Mods -> User Override Files. Redo the Save, Process and Submit steps in your UMUI job.

Check the bld.cfg file in $DATAW/ummodel/cfg to see that the values for tool:fflags and tool::ldflags have changed.

How do I apply a file specific compile override?

The equivalent of the comp_specific file holding standard deck specific overrides is now held in FCM as a config file. If you want to see what it looks like go to:

http://puma.nerc.ac.uk/trac/UM/browser/UM/branches/dev/um/VN7.1_machine_cfg/src/configs/machines/hector-pathscale/overrides64.cfg

The most likely scenario is that you need to change one of these default declarations or define a new one.

In the UMUI panel SubModel Independent -> Comp and Mods -> User Override Files there is a table where user override files can be specified.

For example to override the compile options for sice_htf.F90, create a text file with the line

bld::tool::fflags::UM::atmosphere::boundary_layer::sice_htf %fflags_all -O0

Which redefines the default optimisation for code file sice_htf from -O2 to -O0.

In the UMUI panel SubModel Independent -> Comp and Mods -> User Override Files add the path to your override file to the 'User override files' table.

Redo the Save, Process and Submit steps in your UMUI job.

Check the bld.cfg file in $DATAW/ummodel/cfg to see that your override has been applied.

How do I upgrade a branch from one UM version to the next?

This can be achieved by either using the fcm gui or using fcm on the command line. Instructions for both methods are given below.

Follow the instructions, and change the text delimited with < > as appropriate for your branch:

Method 1: Using the FCM GUI

  1. Change directory to $HOME/branches (or wherever you usually checkout branches to).
  2. Type fcm gui to start up the user interface.
  3. Click on Branch/Create and fill in the following sections:
    • URL = fcm:um@vn<?.?> (replace ?.? with the UM version of the job you want to include this branch in)
    • branchname = <same as last one>, where branchname will appear after the UM version number (e.g. VN?.?_<branchname>
    • branch type = dev
    • source type = trunk
    • prefix option = normal
    • related ticket = <ticket number>
    • Other options = --password "" (this is only needed the first time you create a branch)
  4. Click on RUN
  5. Edit the editor window if you wish, close the editor window and click on 'Yes' when asked if you would like to create the branch.
  6. Click on Checkout
  7. Change URL to the one you just created. This should be in the syntax svn://puma/UM_svn/UM/branches/dev/<your_userid>/VN?.?_<branchname>
  8. Click on RUN
  9. Click on Merge
  10. Change source to point towards the branch you are trying to upgrade (e.g. fcm:um_br/dev/<owners_userid>/VN?.?_<branchname> where ?.? is the version number you are merging up from)
  11. Click on RUN
  12. Click on Commit
  13. Click on RUN
  14. Edit the editor window if you wish, close the editor window and click on 'Yes' when asked if you would like to commit the branch.

Method 2: command line

  1. Change directory to $HOME/branches (or wherever you usually checkout branches to).
  2. Create a new branch
    fcm branch-create -k <ticket_number> <branchname> fcm:um@<vn?.?>

where
<branchname> - same as last one. branchname will appear after the UM version number (e.g. VN?.?_<branchname>)
<vn?.?> - replace ?.? with the UM version of the job you want to include this branch in
<ticket_number> - the number of the related trac ticket

  1. You will be prompted for a commit message. Edit the window if you wish, close the editor and press "Return" when asked to confirm that you wish to create the branch.
  2. Checkout the branch you just created by running: fcm checkout fcm:um_br/dev/<your_userid>/VN?.?_<branchname>
  3. Change directory to the top level of the working copy.
  4. Run fcm merge to merge in the branch you are trying to upgrade. (e.g. fcm merge fcm:um_br/dev/<owners_id>/VN?.?_<branchname> where ?.? is the version number you are merging up from)
  5. Run fcm commit
  6. You will be prompted for a commit message. Edit the editor window if you wish, close the editor and press "Return" when asked if you would like to commit the branch.

How do I create a branch at HG6.6.3 (HadGEM2-ES)?

Creating a branch from the HadGEM2-ES trunk (UM6.6.3) is more complex than for all the other UM versions. (For those that wish to know why .... This is because the 6.6.3 trunk is actually a branch of the main UM trunk. So creating a branch at 6.6.3 actually means creating a branch-of-a-branch.)

In order to make things simpler there is a script called create_HG2_branch on PUMA that you can run to correctly create a hg6.6.3 branch. This script should be in your PATH, however, if it isn't, you can find it in the following location:

/usr/local/bin/create_HG2_branch

When you run this script it will ask you a series of questions. Hopefully the answers to the questions are self-explanatory. For the most part you just need to press <Return> to accept the default or type the branch name and, if applicable, the associated ticket number. There are 2 questions that present a list of options to select from:

What revision is the branch to be created at? Enter the option number next to HG6.6.3.

What branch type do you wish to create? Generally you will want DEV::USER so enter the option number 1. When you're more experienced with FCM you'll know when it's appropriate to use the other options. They are explained in the FCM user guide should you wish to know more about branch types.

How do I resolve a branch/file conflict?

Work in Progress

Decide whether to merge one branch into the other or create a new branch and merge both branches into it.

fcm conflicts

FCM will now run the file editor/difference tool xxdiff. <Explanation of how xxdiff works>

How do I restore a deleted branch?

Copy the revision before it was removed, this will restore the branch as it existed in [REVISION] to the HEAD revision.:

fcm copy svn://puma/UM_svn/UM/branches/path/to/branch@[REVISION] svn://puma/UM_svn/UM/branches/path/to/branch

If you have a working copy of the branch you will need to run fcm update to bring the revision numbers into sync. Or if you have no uncommitted changes just delete your working copy and recheck it out.


Sharing code changes (fcm mkpatch)

How do I apply an FCM patch received from the Met Office?

  1. If you have a directory that contains FCM branches, go to that directory. If not then make a directory called 'branches' and cd into that directory.
  1. Create a new branch either via the FCM GUI or using the fcm commands on the command line.
puma$ fcm branch-create [-k <ticket number>] <branch_name> fcm:um@<revision>

where <revision> is either UM revision keyword (e.g. vn7.1) or a subversion revision number
      <ticket_number> is the number of a Trac ticket you wish to associate with this branch
  1. Make a note of the branch URL that it creates.
  1. Apply the patch/changeset to your branch by running "<path to the patch>/fcm-import-patch <URL>" replacing <URL> with the URL that you noted down in the last step. This will now apply the patch code changes to your branch.

[Alternatively, you can type "fcm checkout <URL>" replacing <URL> with the URL that you noted down in the last step to create a working copy in your current directory. You then apply the patch/changeset to your directory by running <path to the patch>/fcm-import-patch PATH. Where PATH is the location of the working copy you just checked out. This will now apply the patch code changes to your branch.]

  1. During the patch application process the changesets should automatically be committed to the repository. You should see lines like
    Transmitting file data ......
    Committed revision 9.
    At revision 9.
    
    in the output. You can also confirm this happened by running the command fcm status which will tell you which files are different from the repository. If it reports back nothing then the files have been successfully committed. Make a note of the final revision number.

For help with any fcm-import-patch problems see Troubleshooting fcm-import-patch

  1. Open up your job in the UMUI and go to the window FCM Configuration --> FCM Options for the Atmosphere and Reconfiguation. In the table "User Modifications" add your new branch's URL that you got in step 3 (e.g. fcm:um_br/dev/ros/VN7.1_example_branch/src). You will need to add "/src" at the end of the URL. Type in the revision number you got from step 7 in the revision column.

How do I share my changes with the Met Office?

To share your code with colleagues at the Met Office you will need to create what we call an "FCM patch", containing the changes you wish to share. They can then import this patch into their UM repository. See How do I create an FCM patch for instructions on how to do this.

How do I create an FCM patch?

In order to create a patch you will need to identify the branch from which you wish to make the patch and which revisions of the branch you want to be included. To do this go to the "Browse Source" in the Trac system, drill down to the branch and then click the link "Revision Log" near the top right hand side. This will display a list of all the revisions on the branch.

Identify which revisions of the branch you wish to be included in the patch, this can be either a single revision or a range of revisions and then run the following command on PUMA.

fcm mkpatch [--revision REVS] --organisation NCAS BRANCH_URL [OUTDIR]

If a revision is specified with the --revision option, mkpatch will attempt to create a patch based on the changes at that revision. If a revision is not specified, mkpatch will attempt to create a patch based on the changes at the HEAD revision. If a revision range is specified (e.g. 867:871), mkpatch will attempt to create a patch for each revision in that range (including the change in the lower range) where changes have taken place in the BRANCH_URL.

If OUTDIR is specified, the output is sent to OUTDIR. Otherwise, the output will be sent to a default location in the current directory ($PWD/fcm-mkpatch-out/). The output directory will contain the patch for each revision and a script for importing the patch.

The --organisation option causes any Trac links in the commit messages to be modified to include extra information about the original changeset. In the above example; Trac links such as ticket:123 will be replaced with NCAS_ticket:123.

Example:
To create a patch including the changesets 884 to 888 in the branch fcm:um_br/dev/ros/VN7.1_bugfixes_from_ukmo the following command would be issued.

fcm mkpatch --revision 884:888 --organisation NCAS fcm:um_br/dev/ros/VN7.1_bugfixes_from_ukmo vn7.1_bugfixes_patch

This will create a patch in a directory called vn7.1_bugfixes_patch.

All you then need to do is tar up and gzip the directory and send it to the other party.

For further information on the fcm mkpatch command please refer to the FCM User Guide available from the NCAS-CMS website.

fcm-import-patch Troubleshooting

In this section you will find solutions to the most common problems encountered when applying a patch to branch.

can't find file to patch at input line 5

ros@puma$ ./fcm-import-patch /home/ros/temp/VN7.1_patch_test
apply-patch: Applying patch for changeset 8906.
can't find file to patch at input line 5
Perhaps you used the wrong -p or --strip option?
The text leading up to this was:
--------------------------
|Index: control/top_level/scm_shell.F90
|===================================================================
|--- control/top_level/scm_shell.F90    (revision 8912)
|+++ control/top_level/scm_shell.F90    (revision 8913)
--------------------------
File to patch:

Errors of this form usually indicate that there is a discrepency in the directory structure that the patch is expecting and what it has been given. This is often caused by the patch being generated from a directory level other than from the top of the branch.

To find where the patch was generated from open the fcm-import-patch script in your favourite viewer. In the comment header at the top of the script there will be a line like

#   Patch created from Met_Office URL: svn://fcm2/UM_svn/UM/branches/dev/frlk/VN7.1_BLLEVS_fixes/src

This tells us, in this case, that the patch was created from the src directory level rather than from the top of the branch VN7.1_BBLEVS_fixes.

Thus to apply this patch successfully issue the following command:

ros@puma$ ./fcm-import-patch /home/ros/temp/VN7.1_patch_test/src

cp: missing destination file operand after ....…

puma:vidale$ fcm-import-patch VN7.1_dyn-bugfixes
/export/puma/data-01/home/vidale/VN7.1_dyn-bugfixes
apply-patch: Applying patch for changeset 12547.
cp: missing destination file operand after `/export/puma/data-01/home/vidale/fcm-mkpatch-out/12547/1'
Try `cp --help' for more information.

An error of this form indicates that the revision number specified when the patch was created were incorrect. The revision number at which the originating branch was created was supplied as the lower range to the --revision option to fcm mkpatch rather than the revision number of the first commit after the branch was created.

To proceed, you either need to ask the person to create the patch again using the correct revision numbers or alternatively, edit the patch as follows:

Edit the file <path_to_patch_dir>/fcm-import-patch to remove the commands that apply the first part of the patch. Search for a section that looks like the following example.

# Commands to apply patches
${fcm_patches_dir}/12547/apply-patch || exit 1
svn commit -F "#commit_message#" || exit 1
rm -f "#commit_message#" || exit 1
svn update || exit 1

Check that the number on the second line matches the one in the error message you got above (i.e. in this case 12547). If it doesn't then your problem may be different to the one described here - contact the UM Helpdesk for advice.

Delete the 4 lines of code, leaving the comment "# Commands to apply patches" in place.

You can now try re-running the fcm-import-patch script and the patch should now succeed.


FCM Extract ERROR and WARNING messages

How do I know if the Extract process failed?

An error in the Extract process is indicated in the UMUI job submission dialog box with a message similar to

FCM_MAIN: Calling Extract ...
Base extract: failed
FCM_MAIN: Extract failed

To find out what has gone wrong you need to look in the extract output file, which is located in the local extract directory which you have specified in the UMUI panel FCM configuration --> FCM Extract and Build directories and Output levels. For example, if you specified $HOME/um/um_extracts as the root you will get the output in file $HOME/um_extracts/$RUNID/umbase/ext.out. If the message said Model extract: failed you would need to look in $HOME/um_extracts/$RUNID/ummodel/ext.out.

Scroll down to near the bottom of the file and you will hopefully see some informative error or warning message from FCM. See below for some of the most common problems.

vn7.1_cfg: revision keyword not found for .…

Extract command started on Mon Mar 30 14:14:10 2009.
->Parse configuration: start
vn7.1_cfg: revision keyword not found for svn://puma/UM_svn/UM/branches/dev/um/VN7.1_machine_cfg/src/configs/bindings/container.cfg 
in FCM configuration file, abort.

This error in your $HOME/um/um_extracts/$RUNID/umbase/ext.out extract output file means that you are missing the FCM keyword definitions. Make sure that you have the following line in your $HOME/.fcm file:

inc ~um/fcm/etc/um_revisions.cfg

ERROR: UM/control/top_level/um_shell.F90: merge - cannot resolve conflicts:

ERROR: UM/control/top_level/um_shell.F90: merge - cannot resolve conflicts:
  source[0]=svn://puma/UM_TUTORIAL_svn/UM/branches/pkg/Share/VN7.0_machine_cfg/src/control/top_level/um_shell.F90@1018
  source[1]=svn://puma/UM_TUTORIAL_svn/UM/branches/dev/ros/umshell1/src/control/top_level/um_shell.F90@1038
  source[2]=svn://puma/UM_TUTORIAL_svn/UM/branches/dev/ros/VN7.0_example_branch/src/control/top_level/um_shell.F90@1054
  cache0=/home/ros/um_extracts/xdlqt/umbase/.cache/.ext/UM/control/top_level/base__1018/um_shell.F90
  cache1=/home/ros/um_extracts/xdlqt/umbase/.cache/.ext/UM/control/top_level/branch3__1038/um_shell.F90
  cache2=/home/ros/um_extracts/xdlqt/umbase/.cache/.ext/UM/control/top_level/branch4__1054/um_shell.F90

Error messages of this type occur when 2 (or more) branches contain modifications to the same file (um_shell.F90 in this case) and so conflict. The default behaviour of FCM is to fail and force you to resolve the conflict. In this example the conflict is between branches fcm:um_br/dev/ros/umshell1 and fcm:um_br/dev/ros/VN7.0_example_branch indicated by the source[1] and source[2] lines above.

You must resolve the conflict before you can continue. See the FAQ entry How do I resolve a conflict? for instructions on how to do this.

Error: Timed out, host not responding

Sometimes during the extract ssh loses connection with HECToR or more commonly file system access on HECTOR is slow, causing the command to time out. This happens very irratically. The remote command timeout limit has been increased to try and limit this problem. Simply try re-submit the job via the UMUI and you'll probably find the extract will succeed the next time.


Common UM compile output ERROR and WARNING messages

General Guide to UM compile output

See Guide To UM Compile Output

gmake: * No rule to make target <routine1>.done, needed by <routine2>.done. Stop.

An error message like this in a .comp.leave file is likely to be caused by a dodgy preprocessor key somewhere. Check the relevant .f90 file in `<expt_dir>/ummodel/ppsrc/UM/..... directory on HECToR. The name of the file(s) you need to look in will be either <routine1>.f90 or of the form <routine1>-<subname>.f90. If these files are empty then that is the problem.

For example if the error message is

gmake: *** No rule to make target `boundval.done', needed by  `atm_step.done'

Then you need to look at the files <expt_dir>/ummodel/ppsrc/UM/atmosphere/lbc_input/boundval-boundva0.f90 and
boundval-boundva1.f90.

If both of these files are empty then incorrectly set DEFs are the problem. 

To fix this you will need to check what DEFs need to be set or unset for this file to contain code.

Go to <expt_dir>/umbase/src/UM/..../<routine1>.F90 [or <routine1>-<subname>.f90]

At the top of the file you will see something like

#if defined(A14_1B) || defined(A11_2A)

which, in this example, means that for the code to be switched on either section A14_1B (Energy Correction) or A11_2A (Tracer Advection) must be switched on.

Go to the UMUI and switch the appropriate section on.

Re-process and submit the job to recompile.

WARNING: ac.o: duplicated targets for building

Messages of the form:

WARNING: ac.o: duplicated targets for building:
       /work/n02/n02/ros/xdhia/umbase/src/UM/atmosphere/AC_assimilation/ac-ac0a.F90
       /work/n02/n02/ros/xdhia/umbase/src/UM/atmosphere/AC_assimilation/ac-ac1a.F90

are normal and can be ignored. They are a feature of how the compilation and inheritance is set up along with how FCM works.

Makefile:16945: warning: overriding commands for target `gather_field_ml.o'

Messages of the form:

Makefile:16945: warning: overriding commands for target `gather_field_ml.o'
Makefile:16933: warning: ignoring old commands for target `gather_field_ml.o'
Makefile:17110: warning: overriding commands for target `scatter_field_ml.done'
Makefile:17098: warning: ignoring old commands for target `scatter_field_ml.done'

can be ignored. They are a feature of FCM


Trac Questions

How do I receive notification of changes to a branch?

Did you know that you can use an RSS Reader to passively keep up on changes to subversion branches?

RSS - Really Simple Syndication. It is most often used by newspapers to send you their latest headlines - but Trac can also let you know what has been added/updated since you last viewed it.

To use RSS you need an RSS reader. This can be a standalone package or can be done through a Web browser. Below is an example of how to install an RSS Reader extension to Mozilla Firefox. Using Mozilla Firefox

  • Install the Firefox RSS Reader Extension Sage (or your RSS reader of choice - found under News Reading Extensions) This will add the option to view the Sage Sidebar in your browser. You will have to close and re-open Firefox in order to start the extension.
  • From the Firefox menu choose View→Sidebar→Sage from the Firefox menu . This will open a sidebar up in your browser
  • Go to the subversion branch which you wish to follow. Near the top right you will see a link for “Revision Log”. Click that link. Now notice in the bottom left hand corner of your browser an orange icon – indicating it is available as an RSS page. Click on this icon.
  • Now, with your mouse, grab the little icon in the URL address bar and drag it to the Sage Sidebar
  • You will now see in the Sage sidebar on the left of the browser new pages in bold. Click the blue circle button on the top half of the sidebar to refresh updates, and click on the ‘Mark as read’ button - the third from left button on the bottom half of the sidebar - to mark all those pages as read. That way only the pages which are new since you last read will show up in bold.
Last modified 2 years ago Last modified on 08/25/15 12:02:41